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FOREWORD 



This volume contains material which supports and expands the voca- 
tional education “status' 1 discussion in Volume I. Some ot the appendices 
provide supporting material for various sections ci Volume l, while others con 
lain directly relevant information that did not fit directly into the discussion, 
appendices A, B r and E provide supporting information, while Appendices D 
and E contain additional material dealing with unique topics . 
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APPENDIX A 



DOCUMENTATION SUPPORTING THE DISCUSSION OP THE t 

VOCATIONAL EDUCATION SYSTEM 



VOCATIONAL EDUCATION IN PROPRIETARY SCHOOLS 

Private vocational schools offer hundreds of different courses? one 
source quotes 1,483 separate courses of instruction in E44 reporting trade and 
technical schools alone. — However, no single instance has been found in 
which the facilities of a private business school have been utilized to carry 
out a program under a contract authorized by the Vocational Education Act of 
1963.-' Students have been and continue to be trained in private vocational 
schools under other Federal legislation administrated by the following Federal 
agencies: 

• Veterans Administration | 

* Vocational Rehabilitation 

o Manpower Administration '• 



-^A.H. Belitsky, Private Vocationa 1 Schools and Their Students ;: L imited 

Objectives— Unlimi te d Opportunities , Schenkman Publishing Company, Inc. , 
Cambridge, Massachusetts, 1969. : 

2 / 

Testimony of Richard A. Fulton, United Business Schools Association, 
before the House Committee on Education and Labor on H.R. 15066, "A 
Bill to Amend the Vocational Education Act of 1963," March 1968. 
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• Immigration and Naturalization Service 

• Office of Economic Opportunity 

• National Vocational Student Loan Insurance 

• Bureau c ndian Affairs . 

Just as Federal funds come from diverse sources to private schools 
offering diverse programs, the schools themselves are widely different. Some 
require high school graduation or its equivalent for admission (2 32 of 411 trade 
and technical schools in 40 states);- others have no formal educational require- 
ments (49 of the 411 schools) . Some start classes as often as weekly (14 of 83 
schools belonging to the National Association of Trade and Technical Schools 
(NATTS)) ; ^ very few limit starting classes to once or twice annually (9 of the 
83 schools) as is the case in the public school system . Virtually all private 
schools operate on a year-round basis (124 of 128 NATTS schools surveyed by 
Belitsky operated 48 weeks or more per year) , State licensing of proprietary 
occupational schools is found in only 20 states and the regulations vary widely. 
Accreditation of trade and technical schools by a designated agency of the U.S. 
Office of Education was first iniroduced in 1967, 

The costs of instruction in private schools vary substantially. In FY 
1970 the Federal Government expended $20,524,452, under programs administered 
Dy the U.J. Department of Health, Education, and Welfare, Office of Education, 
for 10,866 trainees attending 138 different courses at more than 155 private 
schools, A breakdown of this expenditure into cost per trainee shows the fol- 
lowing distribution: 



Total Federal cost 
per trainee* 


Number of 
courses 


Percent of 
total courses 


Less than $1 , 000 . . 


. 35 


25.4 


$1,000 - 1,999 • . 


. 33 


23.9 


$2,000 - 2,999 . . 


. 30 


31.7 


$3,000 - 3,999 . . 


. 25 


18.1 


$4,000 and more . 


. 15 

138 


10.9 

iFoTo 



*Table drawn from HEW infot mation . 

The average cost per trainee was $1,899. The lowest cost per trainee was $210 
(waiter-waitress) and the highest $8,250 (medical clerk). Since one may assume 
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Belitsky, o£. cit , 



that large numbers of the estimated 1.6 million students attending private 
schools; (1966) do so at their own expense, the question may be asked, why do 
people pay for vocational education when free courses are offered by the public 
school system. A Stanford Research Institute Study limited to Santa Clara 
L ounty , California, reported that: 



Students most frequently mention time, convenience 
and course content in explaining their decision to 
enroll in a proprietary school program. They observed 
that they could usually start class within a week after 
enrolling, and that the course length set completion 
within a relatively short period of time— less than a 
year and often under six months. The/ pointed out that 
the curriculum was entirely skill-oriented and free of 
what they considered to be non-essential subjects. 
Finally, many students mentioned placement service 
believing that the school’s continuation as a commercial 
enterprise would depend on the degree to which its stu- 
dents were^uccessful in securing employment after 
training 



One sfudy of 1,105 students, limited largely to high school graduates taking 
auto mechanics and auto body fender repair courses, indicated 128 dropouts, or 
less tian 13 percent. -S/ Th e same source also reports that of 128 NATTS schooh 
surveyed, 75 percent reported providing placement service to the students while 
they attend school, 99 percent reported provking this service upon graduation, 
and 80 percent reported that thev provide placement service "for life." 



Data are available on a demonstration project of the United Business 



Schools Association (USBA) funded under the I/1DTA program which involved par- 
ticipation of member private business schools in 17 States and the District of 
Columbia to train 1,080 trainees. The purpose of the program was to demonstrate 
the el fectiveness of private school participation in ser/ing the disadvantaged 
and to demonstrate the use and effectiveness of the "individual referral method" 
by local employment services in contrast with the established "class group" or 
"under contract" method of referral. The proc ram was broadened to include mem- 
ber schools cf NATTS, so technical as well a; business courses were included 
in the program. 

Covering the period 15 February 1967 through 31 August 1969 USBA 
reported 50 private schools participating, wivh l,i73 enrollments, 818 trainees 
who lad completed the training objectives, and 355 (30 percent) who had not ^ 



^ Quoted by Fulton in hearings , footnote 2 , supra . 

6 / 

“ / Balitsky, op. cit . 

7/ 

Fata source: Parts I and II of final report prepared by USBA for USOF under 
contract QEC2-7-002930-2930. 
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The average tuition cost per completion was reported as $584 for one program 
and $737 for another. The average tuition cost "per clock hour" varied from a 
low of $0.51 (West Virginia) to a high of $1.03 (Indiana and Massachusetts) . 
Available data on the background of the trainees are sketchy, since not all par- 
ticipating schools reported. However/ from those that did (39 schools) it 
appears that the bulk of the trainees were female, in the 19-34 age group, had 
never been employed before or had been gainfully employed less than 2 years , 
and had completed the twelfth grade prior to enrollment. Many were unemployed 
or underemployed at the time of enrollment and over 200 were reported to have 
been "hard-core unemployed" (27 weeks and over). It is worthwhile to note 
that the first trainee was enrolled in school 35 days after the contract was 
signed* 

Employment data were available on only one program involving 22 7 
referrals who had completed their courses; of these, 165 (73 percent) were 
employed at the time of the report (30 June 1968). The pertinent information is 
as follows: 



Status P lacement Salary (weekly) 



In occupation for 




By employment 




$90 or more . 


. . 15 


which trained. . 


. 128 


service * • * 


33 


$65-90 .... 




In related 




By the 




<$65 


. . 23 


occupation . . . 


. 22 


school • • • 


85 


In nonrelated 




Self 


37 


UnKnown ♦ . . 




occupation . . . 


7 


Other 


2 






Unknown 


8 


Unknown .... 


8 






Total . , 


* 165 


Total. . . 


165 


Total . 


. . 165 



While this is a small sample of the 818 students who completed training in the 
entire program, it indicates that (a) the training is employment-related and (b) 
the school assists in placement. 

A follow-on program was initiated for the period 28 June 1969 to 
31 August 1970 to expand the use of private schools in 26 States and the Dis- 
trict of Columbia to meet two objectives of MDTA:^/ (a) provide upgrade 
training to meet the needs of employers in the community and (b) provide part- 
time upgrade training to create new entry level positions. As of 17 April 1970 
there were 43 participating private schools with 633 enrollments in Project 
Upgrade (147 male, 550 female) and 64 dropouts (9 percent). The average tui- 
tion cost was $356. The trainees were about half white and half Negro (10 



8 / 

Memorandum to: State Directors of Vocationa’ Education, from Howard A, 
Matthews, Division of Manpower Development and Training, 21 November 
1969. 
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American Indian) , and mostly in the 19-34 age group. The great majority of 
the trainees (389) in this program had completed grades 9-11; only 90 had 
completed the twelfth grade; 22 had post-secondary education. 

WORK EXPERIENCE PROGRAMS; COOPERATIVE VOCATIONAL EDUCATION 
AND WORK-STUDY 

Two Important programs in vocational education include work exper- 
ience for students while they are still in school; however, the two have quite 
different objectives. The objective of cooperative vocational education is to 
provide work experience that parallels or supplements classroom activity. The 
objective of work-study is simply to provide income to students v/ho would 
have great difficulty remaining in school without it. 

The material on the two programs is presented in a comparative man 
ner and is organized into three sections: the first deals with the legislative 
background supporting each; the second presents information on funding and 
enrollment; and the final section is a summary that includes some evaluation 
of the programs . 

Background of Legislative Authority 

Work-study and cooperative vocational education programs are not 
new. Work -study programs were authorized in Section 13 of the Vocational 
Education Act of 1963 (PL 88-210, 18 December 1963) a-ong with funds of 
$30 million for FY 19 65, $50 milUon for FY 1966, and $35 million for the two 
succeeding fiscal years. According to the statute, the Commissioner of Edu- 
cation had the responsibility to divide the amounts appropriated between work- 
study programs and residential vocational education schools. 

Cooperative vocational education programs, on the other hand, were 
not authorized specifically in the act of 1963 but were funded as a method of 
education under the broader provisions of Section 4 ("Uses of Federal Funds 1 ') . 
Payments to employers were not authorized. 

In the Vocational Education Amendments of 1968 (PL 90-576, 16 Octo- 
ber 1968) separate parts are devoted to each program (Part G— 11 Cooperative 
Vocational Education Programs" (Section 171), and Part H— "Work -Study Pro- 
grams for Vocational Education Students" (Section 18‘.)). Doth programs are 
aimed toward providing financial assistance that would be of benefit to disad- 
vantaged youth. In the case of cooperative vocational education programs, the 
amendments provided that the State plan shall set forth policies and procedures 
which give assurance that "priority...*" given to areas that have high rates of 
school dropouts and youth unemployme. ." (Section 173, (a)(5). In the case 
of w^rk-study programs, the State plan is to set forth principles for determin- 
ing priority that gives preference to applications submitted by local educational 
agencies "serving communities having substantia! numbers of youth who have 
dropped out of school or who are unemployed"(Section 182. (a)(3) .) 
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Under the 1968 amendments, therefore, funds for both cooperative pro- 
grams and work-study programs are directed toward disadvantaged areas , those 
with high dropout rates and high youth unemployment . Work-study programs 
are directed, in addition, to disadvantaged students . Employment shall be pro- 
vided only to a student who "is in need of the earnings from such employment 
to commence or continue his vocational education program .** (Section 182. (b> 
(2)^P).) This specific orientation to vhe disadvantaged is significant and will be 
returned to later in connection with evaluation of the proarams and their suita- 
bility for expansion. 

Two important changes were made in cooperative programs by this 
legislation. First, provision was made for reimbursement of unusual student 
cost (Section 171); second, provision was made 

. . .for reimbursement of added costs to employers for 
on-the-job training of students enrolled in cooperative 
programs , provided such on-the-job training is related 
to existing carrier (sic) opportunities susceptibly of 
promotion and advancement and does not displace other 
worker- who perform such work . (Section 173. (a) (3).) 

federal funds are authorized for paying all or part of the State's expenditures 
on cooperative programs, and 80 percent of the amount expended for student 
compensation and administration under work-stud*' programs. 

Summary Characteristics of the Two Programs . The different objec- 
tives of the cooperative and work-study programs were noted at the outset of 
these remarks . Attention should be drawn at this point to other differences. 
Payment under work-study is limited to full-time students at least 15 years of 
age ani less than 21 years of age. The amount of payment is limited to $45 
in any month ($60 if the student is not within reasonable commuting distance 
of the school) or $350 in any academic year ($500 if commuting distance is a 
problem), and the employment shall be for the local educational agency or for 
some ether public agency or institution. There is no requirement that the 
employment and the in-school vocational training be related. 

Cooperative programs are available to a broader group of participants 
but require formal ties between school and employer for the provision of school 
training and on-the-job training. A cooperative program is defined as a 

. . .program of vocational education for persons who, 
through a cooperative arrangement between the school 
and employers, receive instruction, including required 
academic courses and related vocational instruction by 



9/ 

~ See U.S. Department of Health, Education, and Welfare, Office of Educa- 
tion, Bureau of Adult Vocational and Technical Education, Division of Voca- 
tional Education, Development Branch, Resource M a nual 71 for the Deve lop- 
ment of Cooperative Vocational Education Programs (draft), 3 August 1970. 



alternation of study in school with a job in any occupa- 
tional field, but these two experiences must be planned 
and supervise d bv the school and employer s so that each 
contributes to the student's education and to his emplo y- 
ability , (Section 175, emphasis added*) 

The program is available to "persons'* in any occupational field and, although 
wage payments for the students come from the employers , Federal funds may be 
used to reimburse employers for added costs. To recapitulate the requirements^- 
an "arrangement" between school and employer must be made whereby both con- 
tribute to planning and supervising the school and work experience. The objec- 
tive is to enhance both the student's education and his employability. 

Funding and Enrollment 

The authorizations and appropriations for the two programs under the 
1968 act (Parts G and H) are as shown in Table 1. 

TABLE 1,— Federal funds authorized and appropriated for cooperative 
vocational education and work-study programs 



Fiscal 

year 


Cooperative programs, 
$ millions 


Work-study programs, 
$ millions 


Authorized 


Appropriated 


Authorized 


Appropriated 


1969 


20 


0 


35 


0 


1P70 


35 


14 


35 


4.25 


1971 


50 




45 




1972 


75 




55 





Source: USOE, BAVTE, Division of Vocational and Technical 
Education, Development Branch, Resource Manual 
71 for the Developme n t of Cooperative Vocational 
Education Programs (draft), 3 August 1970. 



Two observations may be drawn from the data on authorizations and 
appropriations. First, no appropriations were made for FY 1969 and the appro- 
priations for FY 197C were modest in terms of authorizations, particularly for 
work-study programs. The second observation is, given the recent availability 
of limited funds shown, there is little likelihood that programs implementing 
Part G and Part H of the act of 1968 are far enough along at the present time for 
meaningful evaluation of these techniques as spelled out in that legislation. 

As is more fully developed in the next section, the evaluation of these pro- 
grams is based upon what took place under the 1963 act and by analogy to MDTA 
and other manpower training programs. 

7 
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Appropriations, expenditures and enrollments for work-study programs 
are presented for the fiscal years 1965 through 1970, inclusive, in Table 2. 

The data show: 

• Federal expenditures less than appropriations 
in all years 

• Large Federal expenditures in fiscal year 1966 
were followed by increasing State and local 
expenditures in the next 2 years, 

• Enrollments increased more than threefold from 
FY 1965 to FY 1966 and decreased each year in 
the 2 later years . 

• Expenditures per enrollee varied from $153 to 
$316, well below the $350 and $500 limits in 
the act of 1963 (Section 13 * (c) (3)) . 

• No Federal funds were appropriated for work- 
study in FY 19 69; data on State and local expendi- 
tures and enrollments are not available for that 
year » 

Enrollments in cooperative vocational education are available only 
for FY 1 969 (shown in Table 3) and since these programs were not funded sepa- 
rately under the 1963 act as a method of instruction, there are no data available 
on expenditures . Over 90 percent of the enrollments show n are at the secondary 
level; distribution, office, and trades and industry dominate the occupations for 
which training is provided. An important observation, not shown in the table, 
is that the 230,229 enrollments in cooperative programs represent 4.8 percent 
of all enrollments in secondary and pos* -secondary vocational curricula in FY 
1969 (4,785,480 total enrollments). 

Summary and Evaluation 

Work-Study Programs . No evaluations have been found of work -study 
programs in vocational education (for example, the extent to which work -study 
has permitted students to remain in school). In the Annu a l Report on Vocational 
and Technical Education for FY 19 68, the decline in eiiroDments from FY 1967 to 
FY 1963 (from 50,041 to 37,008 shown in Table 2) is reported with the 
remark that "many students needing financial assistance were unable to parti- 
cipate in work-study because of reduction or cancellation of programs-" The 
report goes on to note that the Neighborhood Youth Corps (NYC) vn-school pro- 
gram has objectives similar to work-stud/, but in some cases different require- 
ments for eligibility For example, some (but not all) NYC programs are limited 
to high school students. The report states that the average cost per work-study 
enrollee of the NYC in-school program was $722 (Federal share at $650). Assum- 
ing a vocational student eligible for either program, this suggests he stands to 
receive greater financial assistance under NYC. 
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TABLE 3 . —Enrollments in cooperative vocational education 
by occupational category FY 19 69 



Occupational 

Category 


Secondary 


Post- 

secondary 


Total 


Agriculture. 


6,507 


2,608 


9,115 


Distribution . 


93,351 


6,104 


99,455 


Health 


6,708 


963 


7,671 


Home Economics . 


3,372 


79 


3,451 


Office 


55,668 


5,519 


61 f 187 


Technical .... 


114 


1,229 


1,343 


Trades and 
Industry. 


40, 121 


4,833 


44,954 


Other 


3,053 


— 


3,053 


Total 


208,894 


21,335 


230,229 



Source: Analysis and Reporting 

Planning and Evaluation Branch 
DVTE/BAVTE/OE 
1 September 1970 
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The future use of work-study under tne provisions of Part H of the 
I960 act by students seeking financial assistance would seem to depend, there- 
fore, upon the availability of other programs. If one assumes for FY 1 970 that 
80 percent cf the $4.25 million of appropriated Federal funds was spent, approx- 
imating the experience of FT 7 1967 and FY 1968, and that State and local expendi- 
tures and enrollments were the same in FY 1968 ($3.6 million and $37,008, 
respectively); then, work-study would be able to provide aoproximately $190 per 
enrollee in FY 1970. Since the program experienced a decrease in enrollments 
from FY 1966 to FY 1967 and FY 1967 to FY 1968, when larger amounts were avail- 
able Loth absolutely and on a per enrollment basis, it is difficult to see a vigorous 
future for work-study, 

It should be recognized in this connection that work-study has had an 
"on-again off-again" history. Stable and consistent funding, coupled with early 
availability of funds for disbursement throughout the school year, might reverse 
the trend rhown. Legislative changes that increased the level of payments per 
enrollee would also work in this direction and would very likely increase the 
programs' impact in areas of high dropout rates and high youth unemployment. 

The effect on increasing enrollments would be even more pronounced if competing 
NYC programs were cut back. 

C ooperative Programs * No evaluations of cooperative vocational pro- 
grams at the national level have been found. A cost-effectiveness study of 
selected cooperative vocational education programs as compared with vocational 
programs without a cooperative component was started in the summer of 1970.— 
Unfortunately, the proposal specifies that "no follow-on studies of students will 
be conducted as a part of the proposed research project," nor will any measure- 
ment of the effectiveness of cooperative programs in terms of contribution to the 
employability of participating students (Section 175) be made. 

Cooperative programs are highly regarded. The National Advisory 
Council on Vocational Education reported in 1968 that "the part-time cooperative 
plan is undoubtedly the best program we have in vocational education. It con- 
sistently yields high placement records, high employment stability, and high 
job satisfaction . "Ai/ However, virtually no evidence has been found in either 
Publication 1 ( Highlights and Recommendations ) or Publication 2 ( General Report ) 
of the Advisory Council to support the statement quoted above. Furthermore, 
this language appears only in the highlights document.— 7 



IP/ 

~ U .S . Department of Health , Education and Welfare , USOE , Request for Pro- 
posal No. 70-14, Task 4, awarded to Battelle Memorial Institute in June 
1970. 

^^ General Report of the National Advisory Council on Vocational Education , 
USOE, 1968. ~ ~~ ~ 

12 / 

— The G eneral Report refers to the "proven success of the part-time cooperative 
program" (p. 375) but presents no information ordat3 that would support such 
a conclusion. 
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In the previous section it is noted that there were 230,299 enrollments 
in cooperative programs and that this was nearly 5 percent of all enrollment in 
all vocational education programs in FY 1969. It should be pointed out that 
there are additional students receiving " s imulated" business experience in pro- 
grams in distributive education. Under the 1963 legislation to which these 
figure,, relate, cooperative programs were funded under general provisions as 
a method of education. In USOE's Vocational and Technical Education Annual 
Report j fiscal year 1^68, it is reported that many States adopted the technique 
of ''project training" to increase participation in distributive programs. Prior 
to 1963, these programs were limited to serving "employed" persons. The 
report states: 

In the project method, students participate in super- 
vised and coordinated work-related activities, primarily 
in the school laboratory rather than on the job, and enter 
into a series of contracts with a teacher coordinator 
rather than an employer. 

No separate breakdown of students participating in the project method is available. 
However, the existence of the method and its reported widespread use give evi- 
dence of the merit assigned cooperative programs by the professional vocational 
education community. 

The Office of Education has taken a number of sieps toward the develop- 
ment of cooperative vocational education programs under Part G of the Vocational 
Education Amendments cf 1968. A reading of A Guide for Cooperative Voca - 
tional Education shows a large input of study, analysis, and other efforts on 
the part of vocational educators in the public school system but virtually no 
input from the community of employers. The sole stated contributor from the 
latter group is the vice president for public relations of Montgomery Ward and 
Company. It is also stated in the "Foreword" to the Guide that the National 
Conference participants who prepared the draft document were divided into 10 
task forces, each of which was led by an outstanding vocational educator. 

Thus, the Guide, which was prepared to assist the States in starting 
new programs under Part G, reflects the absence of meaningful participation by 
employers. Evidence of two-sided cooperation between school and employer 
is lacking. For example, less space is devoted to "Reimbursing Employers for 
Certain Costs" (page 63) than is devoted to "Equipping the Coordinator's Office" 
and "Furnishing the Classroom" (page 55). Furthermore, the statement is made 
that Part G provides for reimbursing employers " when necessar y for certain 
added costs incurred in providing on-the-job training through work experience" 



See particularly: Resource Manual 71 , noted above, and University of 

Minnesota College of Education, Division of Vocational and Technical 
Education, A Guide for Cooperative Vocational Education, Minneapolis, 
September 1 969 . 
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(emphasis added). In this connection it is stated In R esource Manual 71 (noted 
earlier) that "payments to employers will be made only when necessary; reim- 
bursing employers fox added costs, therefore, is not to be implemented as a 
standard practice." The "when necessary" language appears in the "Findings 
and Purpose" of Part G (Section 171) of the statute. Section 173 dealing with 
the "Plan Requirement" of a State, requires that the policies and procedures 
adopted " must give assuranc e that provision is made for reimbursement of 
added costs to employers. (emphasis added). Since Part G contemplates 
vocational instruction by alternation of study in school with a job in any occu- 
pational field, and reimbursement for added costs is an obvioi s inducement 
to employers , one may question the wisdom of the vocational education com- 
munity preparing, unilaterally , a guide the States with the interpretation 
slated. As is shown later from the evaluations that have been conducted of 
MDTA and JOBS programs, the question of employer reimbursement (who pays) 
is important because it has a direct influence on who is to have effective control 
of the program. 

The Guide for Cooperative Vocational Education also contains a draft 
contract between school and employer, "Cooperative Vocational Education Train- 
ing Agreement Program" (page 69). The form was adapted from the form developed 
by the Department of Vocational Education, Texas Educational Agency. No infor- 
mation is provided in the guide regarding employer participation in preparation 
of the draft, or the strengths and shortcomings of the text as f een through the 
eyes of potential employer participants in Part G programs. Neither is there 
any evaluation of the proposed language based on the Texas experience with 
the instrument . i| 

;i 

Part G programs in vocational education (a) require employer participa- 
tion for on-the-job training; (b) require a cooperative agreeme nt between school 
and employer; (c) are to contribute to both the education and )'he employability 
of the participant; and (d) are primarily to serve areas that have high rates of 
school dropouts and youth unemployment, A number of MDTA and JOBS projects 
aimed at training the disadvantaged which have cooperative ^elements between 
school and employer and which meet additional requirements jbf Part G, have 
been evaluated.-^/ There is not a one-to-one relationship between these 
"remedial" programs which have been evaluated and the "preventive" programs 
contemplated in Part G. For example, the congressional mar date simply requires 
that State plans give priority to areas that have high school dropout and youth 



14/ 



These include: UBSA contract; Timony Halnon, The Regioj - i iv Pilot Cooper - 
ative Program, Findings and Recommendation s (unpublished report), Division 
of Manpower Development and Training, December 1969; lack R. Grisham, 
Evaluation of the Manpower Development and Training Prc gram in the State 
of Florida FY 1971 ; Evaluation of the 1QBS Program in Min i ? Cities , TM-WD- 
(L) -3 1 3/001/00 , Systems Development Corporation, September 1969; and 
The Job Opportunities in the Business Sector Program, An Analysis of Impac t 
in Ten Standard Metropolitan St atistical Areas , Greenleigh Associates , Inc., 
June 1970« * 
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unemployment rates. The remedial programs are directed toward the "hard-core" 
unemployed, which is an older population. Part G (and prior) cooperative pro- 
grams in vocational education do not require that the persons (as opposed to 
areas) served be disadvantaged. Employers tend to prefer (and teachers tend 
to place) nondisadvantaged students in cooperative jobs. However, to the 
extent that programs under Part G might be expanded, they must necessarily 
dig deeper into the population of disadvantaged areas , and this will tend to 
narrow the gap between +he "argot populaiions of remedial programs and prevent- 
ive programs. The evaluation experience of remedial programs is relevant here. 

Any major expansion of Part G programs will likewise tend to close 
the gap between the "cooperative" employers now participating in cooperative 
vocational education programs and the partic ipants in the remedial programs , 
who tend to be representative of the business community. Employer participa- 
tion under Part G is strictly voluntary. Most participating employers are ser- 
vice oriented, e*g., marketing, retailing, office, and the like. A recent (1970) 
study-^-*' w hich the National Association of Manufacturers participated shows 
that only a token dialogue now exists between industrial employers and the 
public school community of vocational education adminis trators and educators. 
An expansion of programs under Part G, particularly of those which are indus- 
trially oriented, should foster better communication. 



It should also be noted that the remedial programs tend to be massive 
programs, funded at the level of hundreds of millions of dollars at the present 
time . — ' By comparison there is room to expand Part G programs, which were, 
as noted above, funded in the amount of $14 million for PY 1970. 



Given the distinguishing characteristics of the remedial and the pre- 
ventive (Part G) programs, there are at least three findings from evaluation of 
remedial programs that are relevant to Part G programs ard to their expansion. 
The first is that the business community appears reluctant to enter into coopera- 
tive agreements and pay the minimum wage when the standards for :he program 
are not their sole province. This seems to be the case even when employers 
are fully compensated for the services they render. If this attitude is as pre- 
valent as some of the evaluations suggest, the provision in Part G to reimburse 
employers for added cost only "when necessary" and not as a matter of course 
will act as a major limitation on the growth of the program. A large number of 
employers prefer to operate their own training programs rather than "get involved 
in government red tape." Reimbursement for added costs of training and lov’ered 



^^National Association of Manufacturers , "Vocations 1 Education Study-Group 
Discussion Paper," xeroxed, undated. 

-^See: "Remarks ’ by William R. Bechtel, Staff Director, Senate Subcommittee 
on Employment, Manpower and Poverty, 13 May 1970. 
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productivity should provide an incentive to employers, particularly whore the 
in -school training is related to the job. 

The second finding that seems to be common to these evaluations is 
that a successful program requires that funds be made available for transporta- 
tion expenses (to and from the job, the student’s residence, and between school 
and job) and also that funds be provided for all kinds of expenses which are 
usual to the more fortunate but unusual to maiy potential trainees, particularly 
the hard-core unemployed. More than half oi this target population are women; 
thus, lor example, provision for child care is- essential to a successful school- 
job program. In describing the needs of student participants, one study found 
that 

, . . health matters may be more significant in a coop- 
erative training endeavor than in a Jull-time institutional 
situation, , .Individual projects reported other needs . 

Macon reported a range of physical disabilities. Knox- 
ville expressed a need for clothes, especially before 
trainees are expected to be placed in cooperative employ- 
ment. One woman, for example, had only one dress and 
was reluctant to report for work in such condition. . .The 
most serious obstacle to successful cooperative employ- 
ment, apart from employer reluctance, was the fact that 
many if not most industries required some form of physical 
examination as a condition of employment. 

The provision in Part G to pay for only the unusual costs of students, 
unless liberally interpreted, will wo r k to inc ease dropouts and the number of 
skipped classes and job assignments, as we.l as to limit the program's success- 
ful expansion. The statute directs the States to give priority "to areas that have 
high rates of school dropouts and youth unen ployment." The evidence is clear 
that the more this is interpreted to mean give; priority to the hard-core unemployed, 
the greater will be the requiremeni in Part G to interpret "unusual costs" more 
liberally . 

Many other program characteristics: are noted in the evaluations that 
have been found to be associated with succ assful programs: clear understanding 
between school and employer on such matters as the procedure for referrals (not 
mentioned in the draft agreement on page in the Guide for Coopera t ive Voc a- 
tional Education , noted above); flexible scheduling, additional staff training, 
and sensitivity training for employer staff; employing available waivers of child 
lalor and minimum wage laws; and so on. From all that has been reported, one 
adiitional characteristic has been singled out for relevance to the programs pro- 
vided under Part G— the need for pre-vocat lonal training and the associated 
requirement to fit the training period to the needs of the student. The evaluation 
of Florida's MDTA program, cited previourly, addresses the issue of pr*_-voca- 
tianal training: 



-^Halnon, op. clt . 
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Of the total trainees (146) received from 
12-9-68 through 10-31-69, 37% were reading 
below tho 5th grade level, 15% were working 
on or below the 5th grade level in math, and 
16 trainees had a history of mental illness and 
had been in Mental Institutions. . . . Tne pro- 
gram first lacks the needed personnel for test- 
ing, training, and counseling and basic edu- 
catio i . . . It is beyond the abilitv of the 
coordinators to teach specific skills, such as 
Nurse Aide, etc. , and it is impossible; to 
place trainees in On- Job-Training while per- 
forming at the low level . . . Further, it is felt 
that the time limit fer the training of the hard- 
core disadvantaged is unrealistic. If each 
trainee is to progress at his/her individual rate 
until reaching full time employment maturity then 
no time limit should be set. This experience 
yields that out of the 149 trainees referred, at 
least 80% need to go beyond the initial 22 weeks 
and 20% will have early completion. Employers 
are unanimous in their receptivity of the program 
only if the trainees have had some type of pre- 
vocational training in the occupation in which the 
employer will be willing to continue training the 
trainee . 13/ 

This evaluator is saying that successful cooperative programs for the 
disadvantaged require flexibility. They require flexibility in curricula and in 
periods of training. Pre-vocational programs and other special training should 
be provided when needed and bypassed when not. Training periods should 
match the needs of the individual student. Part G programs require the obvious 
flexibility of frequently starting classes, varied curricula, and the 12-month 
academic year that is characteristic of private vocational schools. As showr: 
in Chapter Hi of this report, the public vocational school system generally, 
and USOE in particular, have been reluctant to contract with private vocational 
schools, which offer the flexibility an expanded Part G program will require. 

In summary, the development of successful cooperative programs under 
Part G of the 1968 amendments will depend upon the degree to which the in- 
dividual projects become true joint ventures of employer and school. At the 



w 
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present time there is little evidence that the interests and attitudes of the 
business community or participation by that sector, are reflected in the plan- 
ning undertaken by che Office of Education. In addition, application of the 
program to disadvantaged students will be limited by the statutory provisions 
and administrative interpretation that curtail payments for added costs to 
participating employers* calling for reimbursement only "when necessary" ; by 
the statutory limitation on payments to students for only ’’unusual" service 
costs (unless "unusual" is liberally interpreted); and by the failure to use 
private vocational schools that offer additional required flexibility. 

EXEMPLARY PROGRAMS AND PROJECTS 

Introduction 

Exemplary programs and projects provide a bridge between innovative 
occupational programs or projects that sh)W promise from earlier research, 
and established vocational education programs. Since they are designed to 
serve as models for widespread adoption, the Federal role is to be a catalyst 
for development and testing of such programs and to disseminate the results 
of these tests whether they are Federally funded or not. 

The discussion here is broken in three parts. The first deals with 
the legislative authority for exemplary programs; the second with the adminis- 
tration of these programs by the Office of Education. The final section con- 
tains an overview on legislative and administrative strengths and weaknesses, 
and an evaluation of the Federal role. 

Legislative Intent, Authority, and Funding 

The Vocational Education Act of 1963 (PL 88-210, 13 December 1963) 
provided that a State's allotment may be used, in accordance with its approved 
State plan, among other things, for "special demonstration and experimental 
programs ..." (Section 4. (a)(6).) Identical language appears in the Voca- 
tional Education Amendments of 1968 (PL 90-575, 16 October 1968) under a 
new heading, Part B— "State Vocational Education Programs" (Section 122.(a)(8/ 
Although congressional authors itlon for exemplary programs and projects is, 
therefore, not new, the intent of the later act is to increase activity in this 
area at both the Federal and State levels. 

The 1968 amendments contain a lew part devoted entirely to the sub- 
ject: Part D—^Exemplary Programs and Projects . " Separate authorization of 
funds is provided: $15 million for FY 1969, $57.5 million for FY 1970, and 
$75 million for each of the two succeeding fiscal years (Section 142. (a)). 

From these sum?, which are apportioned to the States by formula in the legis- 
lation, the Commissioner of Education is authorized to make half of the grant? 
and the State boards to make half. There is no Federal-State matching require- 
ment for grants or contracts made under this part of the amendments. 
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Part D contains a statement by the Congress of findings and purpose 
(Section 141): 

The Congress finds that it is necessary to 
reduce the continuing seriously high level of yout h 
unemployment by developing means for giving the same 
kind of attention as is now given to the college 
preparation needs of those young persons who go on 
to college, to the job prep aration needs of the two 
out of three young persons who end their education 
at or before completion of the secondary leve l, too 
many of whom face long and bitter months of job 
hunting or marginal work after leaving school. The 
purposes of this part, therefore, are to stimulate, 
through Federal financial support, new ways to create 
a bridge between school and earning a living for young 
people, who are still in school, who have left school 
either by graduation or by dropping out, or who are 
in postsecondar y prr grams of vocational preparation, 
and to promote cooperation between public education 
and manpower agencies, (Emphasis added.) 

Section 141 has been set forth in its entirety and will be referred to in the 
discussion of administrative implementation of the legislation. Part D lists, 
in addition to the quoted section on findings and purpose, seven areas in 
which grants or contracts maybe made (Section 143. (a)(2).) Funds may also 
be used fox planning and development in these areas, and for "establishing, 
operating, or evaluating exemplary programs or projects designed to carry out 
the purposes set forth 1 ' in the statement of findings and purpose. 

Administration of the Legislation 

Administrative responsibility for Part D— 11 Exemplary Programs and 
Projects' rests, at the Federal level, with the Pilot and Demonstration Branch, 
Division of Vocational and Technical Education, Bureau of Adult, Vocational 
and IJbiary Programs, U.S. Office of Education. At the State level this responsi- 
bility rests with the State boards. The first appropriation under the legislation 
initially was for $13 milion in FY 1970, 50 percent lo the States and 50 percent 
to the Commissioner . The latter share was subsequently reduced by 10 percent. 
The anticipated appropriation for FY 1971 is $16 million. 

Programs and Projects Dire ctl y Funded by State B oards . There is one 
principal Federal constraint, in addition to the "uses of funds" provision in the 
legislctlon, on the use of the 5 0 percent of appropriated funds by the State 
boards. Every vocational education exemplary program and project must fit 
into the State system, i.e. , it must be consistent with the State plan, which 
must be approved by the 1J.S. Commissioner of Education. The language of 
the plans is sufficiently general, however, and the reporting of data sufficiently 
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incomplete and non-uniform, that it is difficult to know in any detail in Wash- 
ington what takes place at the State level. A list of the titles and funding of 
the exemplary programs and projects directly funded by the State boards is com- 
piled from information copies of all proposals approved by the boards. Pre- 
sumably, additional information about State programs and projects could be ob- 
tained by request of the U.S. Office of Education regional offices or by request 
of the research coordinating units (RCUs), one of which is located in every State 

The State-administered exemplary projects in vocational education vary 
widely in content and funding. In most cases more than one project was funded 
in each State. The project receiving the smallest amount of funding {$280) was 
for "Office Simulation" (Portland High School, Oregon); the project receiving 
the largest ($116,278) was for "Career-Centered Curriculum for tne Vocational 
Complexes" (State Board of Education, Mississippi). 

Pr ograms and Proje ct s Funded by the Commissioner . State boards or 
local educational agencies seeking their share of the 50 percent of appropriated 
funds allocated to the Commissioner must follow the procedures administered 
by the Pilot and Demonstration Branch of USOE. Proposals to be funded must 
describe programs or projects that combine five aspects in a single operational 
setting: 

• Occupational orientation at the elementary and 
secondary school levels 

• Work experience, cooperative education and 
similar programs 

• Specific training in job entry skills, just prior 
to leaving school, for students not previously 
enrolled in vocational programs 

• Occupational guidance and counseling, and 
initial placement during the last years of school 

• Provision for follow-on funding from regular 
sources . 19 / 

The proposals received are read and evaluated in terms of responsive 
ness to the above requirements. There are 15 outside readers of vocationa’ 
education exemplary proposals all of whom, with a single exception, are <■ 
volved in the administration of vocational education at the State and locai 
levels, either as principals of vocational schools r r State officials in cha 



1 9/ 

— ' USOE, Bureau of Adult, Vocational and Library Programs, "Policy Paper," 
AVL 770-1, 2 October 1969. Manual: Instructions and Procedures (draft), 
1 November 1969, contains specifications for proposing and conducting 
exemplary programs and projects. 




19 



27 



of vocational education programs. Each proposal is read by no fewer than five 
people, including outside and in-house personnel. The 1 January 1970 deadline 
for receipt of proposals v/as waived fv.r Model Cities in 16 states, in an effort 
to tie some specific e emplary projects with the Model Cities program. 

The 1970 goal of the Pilot and Demonstration Branch is to have one 3- 
year project (annually funded) operating in each State and territory. At the 
present funding Level no "new starts' 1 will be considered where there is an on- 
going program, except as a substitute for that program. The "Status Report of 
Part D Proposals (8/24/70)" shows that 168 proposals had been received ill 
disapproved by States) from the 56 States and territories, and that 31 projects 
had been approved. In addition to the proposals themselves, management records 
are kept showing summary information about "Exemplary Projects in Vocational 
Education" (grants made) and thp "Status of Projects in States Not Yet Approved 
(DVTE/PDB 9/10/7 0)." The amount of funds provided (1 year) when grants had 
oeen made varied from $101,049 (Wyoming) to $163,118 (California). As noted 
earlier, the State allocation is determined by legislation. 

Projects funded by the Commissioner will be subject to yearly monitoring 
trips by personnel from the Pilot and Demonstration Branch and visits by personnel 
from the appropriate regional office on an "as needed" basis. In addition to 
quarterly progress reports, the grantee is requirec to prepare an interim report 
at the end of each 12-month funding period and a final report at the end of the 3- 
year project period. The proposal itself must include a plan, to be carried ouc by 
a third party, for evaluating the effectiveness of the orogram or project. A guide 
for authors in preparing evaluation reports is avai able . AQ/ 

The following abstracts of funded exemplary projects were taken verbatim 
from the script of an address given by Dr. Albert ; . Riendeau, chief, Pilot and 
Demonstrations Branch, at the National Institute on Exemplary Projects in Voca- 
tional Education held at Squaw Valley, California, 19-23 July 1970: 

Colorado— The Aims Junior College District, formally 
approved in the State as an area vocational school, will 
provide among the several exemplary components, peer 
counseling for disadvantaged Mexican-A merican students 
who are potential or actual dropouts fron the secondary 
schools. The plan calls for working witi the entire family 
unit, with special efforts to be aimed at working with the 
father. Individual learning packages ani intensive tutorial 
assistance will be provided these students. 
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Massachusetts-The New Urban League of Greater Boston, Inc, , 
will implement the Exemplary Program through the use of a Con- 
tinuing Education Center. Unique among the activities at the 
Center will be the a dvocate and BLack Exemplar roles to be play- 
ed by counselors. The focus is on inner city people, mostly 
Biacks . Incorporated into the design of the Boston project are the 
development of minority exemplars , occupational information, 
attitudinal change, parental involvement, skill training, and 
task analysis. This ExempLary Program reflects an effort on the 
part of a nonpublic sector group to provide realistic innovative 
action in education to meet a serious need. 

Virginia-Called the DILENOW1SCO Four I's Project (for Inter- 
vention, Introduction, Investigation, and InvoLvement) , the 
applicant agency is a consortium of five lacal school divisions 
headquartered in Wise, Va. Located in an area of high unem- 
pioyment the program is designed to intervene in the lives of 
a selected group of youths by i ntroducing them to a broad range 
of occupational information; making it possible for them to 
investigate several occupational areas, they will become in- 
volved in actual work and learning experiences. The target 
group is largeLy potential dropouts. 

Nevada-The Washoe County SchooL District, with offices in 
Reno, developed an Exemplary Project which introduces new 
elements of vocational education at the elementary, secondary, 
and post-secondary levels and combines them with existing 
elements to form a smooth, sequential program. The new ele- 
ments are occupational orientation at the elementary and junior 
levels, and a heavy concentration of counseling, job orientation, 
and placement at the high school level, A health occupations 
curriculum at the senior high school Level is being tried in this 
program also, 

Pennsylvania— The Pittsburgh Public Schools will, for the 7th 
and 8th grade orientation procram, utilize the facilities of a 
renovated elementary school. Students will be rotated for 
career orientation and exploration. With a centralized loca- 
tion for occupational orientation, the Pittsburgh School District 
feels it can provide a greater variety of materials and equip- 
ment as well as keep them current at a more reasonable cost. 

The report requirements imposed on the grantee were noted earlier. The Manual 
of Instruction and Procedures (Section 6; page 5), directs the proposer to de- 
scribe how the results of his project are to be disseminated and to indicate the 
steps that will be taken to make materials, techniques, and other outputs of the 



project available toothers (Manual, Section 6, pacje 5). Presumably, descriptive 
materials will be available through the Educational Resources Information Center 
(ERIC) system. 

Overview: Legislative and Administrative Streng th and Weakness; Evaluation 
o f Federal Role 

At present there are no results available with which to measure or 
evaluate exemplary programs funded under Part D of the 1968 amendments. The 
only readily available sources of information about the programs and projects 
directly funded by the State boards are copies of the proposals; the amount of 
the funding granted is known. Not all of the States have had the projects they 
proposed funded by the Commissioner. Furthermore, not one of the programs 
funded has been in operation for more than a few months. One may question, 
however, whether Part D will result in more than a mere continuation of ’’special 
demonstration and experimental programs” initially authorized under the 1963 
act and now carried forward in Part B of the amendments. There are both legis- 
lative and administrative considerations that detract from a vigorous exploitation 
of exemplary programs and projects under part D. 

The "Rules and Regulations" published by the Office of Education under 
Title 45“"Public Welfare r " Part 1 03-"Research anti Training, Exemplary, and 
Curriculum Development Programs in Vocational Education " 11 / contain deletions 
of the language which appears in the congressional statement of "Findings and 
Purpose" in the 1968 amendments (set forth above in its entirety). The language 
deleted in the "Rules and Regulations" is that referring to reducing the seriously 
high level of youth unemployment by developing means for giving the same kind 
of attention to the needs of this group as is now given to the college preparation 
needs of young persons who go on to college. Two out of three young persons end 
their education at or before completion of the secondary level. This language is 
deleted not only from the published "Rules and Regulations," but also from the 
"Policy Paper" and the Manual: Instructions and Procedures (cited previously) , 
which were prepared by USOE as guidelines for the submission of proposals on 
exemplary programs and projects to be funded by the Commissioner. 

Thus a State board or local educational agency desiring to participate 
in the Commissior 's allottment under Part D cou.d learn only by reading the 
act that Congress anted the m eans developed for giving the same attention to 
the job preparation needs of those ending their education ai or before the second- 
ary level as is nov/ given to preparing young people for college. None of the 
material made available to assist such agencies in writing successful proposals 
to be funded by the grantor Office of Education provides this information. 
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If Congress really intended to offer a new development of the means 
to put the educational experience offered youth not college bound on the same 
basis as that offered the college bound, a case can be made that the Office of 
Education has constricted this purpose severely. Furthermore, in the amendments 
Congress aims at stimulating new ways to create a bridge between school and i 
earning a living for young people; the word '’new" is dropped from the text in 
USOF 1 s manual . 

The Office of Education has determined that exemplary programs shall 
not involve original research and developmental activities but must be based upon 
prior research and development . jL?/ As notes earlier the Office of Education has 
determined that all exemplary programs funded by the Commissioner must contain 
elements of each of four provisions (plus a fifth relating to follow-on funding): 
occupational orientation, work experience, specific training in job-entry skills, 
and occupational counseling. Special searches of the ERIC collection have been 
prepared, one of which summarizes the research and provides a bibliography in 
each area , 

A fundamental question, therefore, is whether these four provisions will 
accomplish the findings and purpose set forth in the legislation for Part D. One 
study of 200 programs in vocational education (McCollum et aL , 1968) aimed at 
disadvantaged students in secondary schools identified and described 25 elements 
that contributed to the effectiveness of the programs studied. ^2/ The four pro- 
visions required by the Office of Education for all exemplary programs were in- 
cluded in the list (items 6, 7 , 11 , and 14). The study report comments that 
further study and analysis directed toward verifying the four provisions in terms 
of accomplishing the purpose of the legislation is not possible. It is also re- 
ported in this study that the cost-effectiveness of the various programs could 
not be evaluated in a way chat would bear up under criticism and that the rela- 
tive effectiveness rank could not be determined for any of the programs. 

Clearly, grantees must report more complete cost and benefit data, on a 
consistent and continuing basis, if the effectiveness of alternative programs is 
to be determined. 

The small amount of funds actually appropriated compared to the funds 
authorized by the legislation will act as a limitation on the effectiveness of 
Part D programs. The anticipated appropriation for V Y 1971 ($16 million) is less 
than one-third the amount authorized by Congress for the second year of the 
program ($57.5 million). One may question, for example, how much impact the 
avaiLability of $153,118 per year for each of 3 years for a pilot project can have 
on vocational education in a .State with a school population the size: of California's. 

22 / 

USOE, "Sources of Information on Prior Research and Development Projects," 
undated , 

2 3/ 

— ' John W, McCollum et al . , /.n Identification and Analysis of Effective Secondary 
L evel Vocational Programs for the Disadvantaged^ Social, Educational Research 
end Development, Inc. , Silver Spring, Maryland, December 1 968. 
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Based on m reported enrollment of 1,036,086 in vocational education classes for 
FY 1963 , — * this works out to slightly less than 15<£ per student per year. Of 
course a State may match the share allocated by the Commissioner with its own 
share, and perhaps add funds obtainable under other programs. This was done in 
the District of Columbia to Implement "A Plan for Career Development"!!/ using 
the career cluster concept. The program is funded at more than $500,000 per year. 

The available funds for an exemplary program could be concentrated in 
a school complex, for example -four elementary schools {1,200 students), two 
junior high schools (2,400 students) and one senior high school (2,000 students). 

In such a pyramid in California, the 5,600 students would each be recipients of 
about $27 in exemplary funds from the Commissioner's allocation along. If one 
further assumes that the pilot program can be offered at 75£ per student-contact 
(or classroom) hour, this would provide approximately 36 hours of orientation, 
work-study, job-entry instruction, or counseling per school year. This is less 
than 1 hour per week. 

There is evidence that these resources would not be adequate for a 
truly effective exemplary program. A recent study of selected exemplary programs 
for the education of disadvantaged children (preschool through grade 12) was based 
on 11 schools that had realized measured benefits of cognitive achievement.!!/ 
Although most of the programs focused on reading and math rather than on the four 
provisions relating to vocational education noted earlier, it is relevant for our 
purposes that the students' exposure to the exemplary program or project was 
never less than 2 hours per week and in nany cases it was for periods twice 
that long. The question may fairly be put whether it is possible to expect much 
in the way of "new bridges" and developing means to put our youth who are not 
bound for college on a par with those who are, given the level of effort that has 
been provided. 

A final w ^rd about evaluation of the twofold Federal role-au a catalyst 
for development and testing of exemplary programs and as a disseminator of the 
results of sue), programs and tests. There is no question that the Office of 
Education has embarked upon a number of activities devoted to accomplishing these 
objectives in implementation of Part D of the 1968 amendments. It is too early 
to tell, however, how well the procedures and techniques being employed will work. 
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USOE, Vocational and Technical Education Annual R eport-Fiscal Year 1 968, 
U.S. Department, of Health, Education ^nd Welfare, Table 1, p. 125. 

In Task Force Report on Vocational Education! May 1969 . 

David G. Hawkridge et al . , A Study of Further Sel ected Ex emplary Programs 
for the Education of Disadvantaged Children, American Institutes for Research, 
Palo Alto, California, June 1969. 
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Perhaps a key issue is whether the four criteria for the funding of 
exemplary programs and projects comprise the most efficient {cost-effective) 
way of accomplishing the objectives of Part D. Data of sufficient quality to 
serve as the basis for evaluating, comparing, and selecting preferred exemplary 
programs and projects from among the many alternatives have not been ;und. 
Further, it does not appear that the required reporting by grantees will guarantee 
the availability of this information in the future* 



APPENDIX B 



DOCUMENTATION SUPPORTING THE DISCUSSION OF 
MEASURING ACHIEVEMENT OF OBJECTIVES 
IN AN ECONOMIC SENSE: COMPARING 
THE EFFECTIVENESS OF DIFFERENT 
OCCUPATIONAL PROGRAMS 



INTRODUCTION 



This appendix, a supplement to Chapter VII of the report, it; 1 
with the cost-effectiveness assessment of the major existing vocatiu 
tion programs (curricula). It will be apparent as the analysis is dev-. ’I- 
the recommendation for a major study in this area is fully warranted, 
data available at the Federal level are State aggregates; the few effe : : 
measures are included, and there are no control group comparisons, 
systematic analysis similar to thai which exists for the manpower proa 
not been developed for the evaluation of different types of occupation 
ing , simply because of the lack of accurate, disaggregated data to .1 
comparisons . Lack of data has also precluded otner crucial compari 
as work/study versus cooperative programs and urban-rural-suburban < 



BACKGROUND 

The flow and disposition of vocational education students a 
in Figures 1 and 2, Figure 1 applies to secondary school and Figure 
secondary schools. 

These flow charts were developed to serve several purpose : 
they indicate *he kinds of data provided by the annual vocational edv 
reports, which are available for program analysis. The cost-benefit 
tions discussed in this section were based on data available throu ; 
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Figure 1 . —Disposition of vocational education completers— secondary schools, FY 19 69 . 
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Figure 2 . — Dispos ition of vocational education completers— post-secondary schools, FY 1969. 



reports. The figures reported for FY 1969 are presented in the flow charts. 

The charts also indicate the items of data used in developing evaluative mea- 
sures, such as the cost per placement and the percent of graduates placed in 
jobs related to their training. 

The cost-benefit ratios developed in this section we re limited by the 
extent and quality of reported data. For example, the only follow-up made 
was in the October following graduation. A longer span of observation would 
have provided more meaningful data. Further, no income information was avail- 
able; thus, as discussed later, it is assumed in the calculations that all jobs 
obtained that relate to training are of equal value. Data on income differentials 
could have provided a basis for weighing the relative worth of the different 
types of jobs obtained. 

EFFECTIVENESS 

There are many different instructional programs provided by secondary 
and post-secondary schools. Included are such diverse courses as automobile 
repair, cosmetology, baking, and appliance repair. These courses are classi- 
fied by the U.S. Office of Education into a system of occupational programs: 
agriculture, trades and industry, office, technical, distributive, home economics 
(useful and gainful), health, and “other." 

Vocational education data for each of these programs are collected and 
reported annually by the States. The data pertain to the financing of programs, 
the instructional activities carried out, and the success of graduates. The 
reported data show a wide range of success across programs. This variation 
occurs in the costs of providing the instruction as well as in the results thut 
are being achieved. 

The graduate follow-up date, reported provide information on the rela- 
tive effectiveness of the different occupational programs in terms of finding 
graduates in jobs related to the ‘raining in October following graduation. In 
the secondary schools, distributive programs had M percent of their graduates 
placed in related jobs in FY 1969; this is the largest percentage for any program. 
The lowest percentage of graduates placed in training-related occupations was 
for home economics (gainful) —Z4 percent for FY 1959 . These are median values 
derived from State data. The median for 3 11 secondary occupational training was 
36 percent. For post-secondary programs, the average percentage of completers 
that obtained related jobs (65 percent) was almost twice as high as that for 
oOuonda.y programs (36 percent). The Table 4 shows the figures for each 
program, together with its dollar magnitude. 
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TABLE 4 . — Percentage of completers obtaining jobs in fields 
related to instruction, PY 1969 









Secondary 


Post-secondary 


Program 




Number of 
Completers 


Percentage^/ 


Program 
slze^/ 
($ millions) 


Percentage^/ 


Program 
s ize^/ 

($ millions) 


Distribution . . 




96.9 


41 


32.6 


50 


11.0 


Office . . . 




425.3 


35 


12G. 8 


57 


44.4 


Health . 




14.4 


35 


9.7 


84 


40,7 


Trades and 
Industry . . 




184.5 


35 


137.1 


59 


67. S 


Agriculture . 




110.2 


32 


86.1 


48 


7.9 


Technical . 




12.4 


27 


11.4 


56 


63.0 


Hone econ 
(gainful 




227.7 


24 


7.3 


50 


3.3 


AM programs . 






36 




6 5 





— Median values for State programs . 

2 / 

— Includes Federal, State, and local funds. 



These figures represent the national experience. There is also con- 
siderable variation among States . Since the data are received separately, it 
is possible to examine each State individually. Not discernible from Federal 
reports, however, is the variation that also exists among the different school 
districts within States, Table 5 shows the amount of variation in program 
success (as indicated by percent of placements in training-related jobs) among 
States . 

It is evident from these figures that the range of State experience is 
considerable. Nevertheless, the average deviations are moderate, showing 
definite clustering around the medians . The important point here is that even 
though the secondary distributive program is shown to be the most effective 
in the percent placed in related jobs (see Table 4), some states (Utah at 6 per- 
cent) have done considerably worse than the median. 
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TA 3LF. 5.— Variation among States in percent of graduates 
obtaining training-related fobs 



Program 


Secondary 


Post-s 


econdary 


Range, % 


Avg deviation 
from median, % 


Range, % 


Avg deviation 
from T lediaS, % 


Distribution . . . 


6-68 


9 


7-100 


17 


Trades and 










Industry . . . 


7-76 


11 


0-88 


14 


Office 


4-84 


14 


13-100 


19 


Health 


0-100 


19 


24-100 


12 


Agriculture . . . 


3-100 


10 


0-100 


24 


Technical ... 


4-100 


14 


11-100 


17 


Home econ 










(gainful) . . . 


5-100 


14 


0-100 
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COST 



The degree to which an objective, such as placement in jobs, is 
achieved can only be regarded as part of an evaluation. There is also 
the consideration of cost, for it may be that higher placement ra^es are 
being achieved at greater expense. 

The program expenditures submitted in the annual reports do not 
represent the total costs incurred for the students. Only the expenditures 
directly associated with vocational instruction arc included, and this has 
the effect of bringing the program cost to a level substantially below the 
total per student cost incurred. Nevertheless, for purposes of making 
comparisons among the different programs, the use of program costs is 
appropriate; the cost-effectiveness ratio (using job placement in training- 
related fields as the measure of effectiveness), rather than being viewed 
in absolute terms, can be viewed as an index of the comparative worth of 
programs. Estimates of cost-benefit ratios developed on the basis of 
estimates of full costs are presented later. 
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RELATED COST-EFFECTIVENESS MEASURES 



The figures shown in Table 6 were calculated by dividing tota’ 
expenditures (Federal, State, and local) by the sum of the number of gradu- 
ates placed directly -n their fields of training plus the number placed in re - 
lated fields. This calculation results in ? cost per placement in training- 
related jobs, which in effect is a cost-efrectiveness ratio. The flow charts 
shown earlier (Figures \ and 2) identify the elements used in the calculation of 
this measure. The introduction of cost into the comparison of effectiveness 
significantly alters the ranking of programs within secondary and post-sec- 
ondary vocational education. It does not, howeser, alter average placement 
rate and cost between the two (65 percent versus 36 percent, and $2,400 versus 
$1,350). At the secondary level , where the distributive program was the most 
effective program in terms of percent of students placed in training-related jobs, 
office occupational training becomes the most cost-effective. Home economics 
was the least effective in placing students, but or. a cost-effectiveness basis, 
home economics occupies fourth place, which represents a considerable improve- 
ment in performance . 

There is considerable variation among the different States in cost-ef- 
fectiveness ratios, just as there was in the components of the ratios (expendi- 
tures and percent placed in related jobs) , The range of the ratios is such that 
there is considerable overlap — some agricultural programs, frv example, are 
more cost-effective than some office programs, even tnough these two programs 
are at opposite ends of the scale, on the average. Table 7 presents data on 
the dispersion around the median of State cost-effectiveness ratios for the various 
occupational programs. 

Thus far, no adjustments have been made in the data. One adjustmen 
could be made for graduates who do not make themselves available for employ- 
ment . A substantial number of vocational education graduate? do not enter the 
labor force. In 1969, 5 1 percent of the graduates of secondary programs could 
not be found or did not make themselves available for employment for a variety 
of reasons. Some went on to further education (25 percent) and some entered 
the armed forces (5 percent); there were 16 percent whose status was not known. 

There is a question as to whether the cost-effectiveness calculations 
should not exclude the graduates who do not make themselves available for 
employment. As a consequence, Table 8 was prepared to present ratios based on 
costs reduced by a proportion equal to the proportion of graduates that did 
not enter the labor force. 

Deletion of costs for non-entrants reduces the cost per pla«. ement 
for secondary programs by about one-hall and for post-secondary programs by 
about one-third. This adjustment does not change the relative positions at 
ihe post-secondary level at all, and causes only minor shifts at the secondary 
level . 



TABLE 6 . — Program costs per placement in job 
related to training, FY 1969 





Secondary 


Post- secondary 


Program 


Program 

costs!/ 

($ per place- 
ment) 


Program 

size?/ 

(5 millions) 


Program 

costs!/ 

{$ per place- 
ment) 


Program 

size// 

($ millions) 


Office , . . . 


868 


126.8 


1,906 


44.4 


Distribution . . 


925 


32.6 


2,019 


11.0 


Health .... 


1,324 


9.7 


1,225 


40.7 


Home econ 

(gainful) . , 


1, 533 


7.3 


1,916 


3.3 


Trades and 

industry , , 


1,944 


137.1 


2, 592 


67.5 


Technical . . . 


2, 728 


11 ,4 


4,208 


63.0 


Agriculture . , 


2, 893 


86.1 


5, 061 


7.0 


All programs 


1,349 




2, 397 





Median values for State programs. 



2 / 

^includes Federal, State, and local funds. 




34 

41 



TABLE 7 • — Variation in cost per placement among States 





Secondary 


Post- 


-secondary 


Program 


Range 


Avg . deviation 
from median 


Range 


Avg . deviation 
from median 


Office 


$30-6,783 


$395 


$7-18,025 


$1,702 


Distribution . . 


455-2,904 


289 


70-15,073 


2,567 


Health 


301-13,933 


925 


547-11,812 


687 


Home Econ 










(gainful) . . . 


356-13,739 


1,059 


52-20,229 


2,018 


Trades and 










industry .... 


844-7,435 


833 


543-13,298 


2,032 


Technical . . . 


163-15,717 


3,613 


333-20,422 


3,230 


Agriculture . . . 


715-16,647 


1,618 


1,061-5,030 


2,329 
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TABLE 8* —Cost per 11 successful" placement adjusted for 
graduates not entering the labor force 



Program 


Cost per placement 


Secondary 


Post-secondary 


Office 


491 


$1, 135 


Distributive 


491 


1 r 855 


Home econ 






(gainful) 


708 


1,476 


Health 


741 


1,062 


Trades and 






industry 


975 


1,037 


Agriculture 


1,203 


2,206 


Technical 


1 ,504 


2,482 



) 
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Since suitable income information by occupational program is not avail- 
able on a national basis, it would not be possible to weight the ratios by ex- 
pected earnings, although such analysis would be useful for national planning, 
to make clear the sensitivity of the rankings to weighting, according to ex- 
pected incomes . 

Finally, with the wide dispersions that exist, there is a question re- 
garding the significance of "average" ratios. In other words, does the variation 
(i.e., range) within programs explain the variation found between programs (e.g., 
office versjs agriculture)? To answer this question, the differences in the 
means of ihe ratios for the various programs were tested for statistical signi- 
ficance by analysis of variance, and the differences were found to be significant. 
Variations within occupational programs cannot explain the differences observed 
among programs , nor can they be attributed to chance* From, a program evalu- 
ation standpoint, therefore, it can be inferred that the different occupational t 
programs are indeed producing different results. 

POLICY IMPLICATIONS OF THE RELATIVE RANKINGS 

Having determined that the apparent differences in results of the 
various occupational programs are real, what are the policy implications? 

There is no question but that it is a policy to maximize the benefits from the re- 
sources being expended for vocational education systems. A system is functioning 
optimally when the marginal returns for each element or each program are equal . 
Although the calculated cost-effectiveness ratios are averages, there is evidence 
that in the case of vocational education, averages are close approximations of 
marginal values. Bringing a system of occupational programs into equilibrium, 
therefore, would necessitate a gradual substitution of funds from programs with 
higher costs per placement to those with lower costs until the ratios are equal 
for all occupational programs. Because of the variations among States, this sub- 
stitution process should take place within the States and should involve reallo- 
cations of funds from all sources —Federal , State and local. Such reallocations, 
or substitutions, would have to be done gradually so that the impact at the margin 
could be measured and kept under surveillance. 

DATA QUALITY 

As mentioned earlier, these cost-effectiveness ratios were developed 
from reported data; obviously, therefore, the accuracy of the reports is an 
important consideration. Three distinct approaches toward assessing the data 
were taken. On the basis of these tests, it was our conclusion that it would nc: 
be possible to claim that the data reported are without validity. 

First, we examined data over a longer time span to see if there was 
stability in the cost-effectivcne ss ratios. If the data had not been valid, wo 
could have expected to find erratic changes in the ranking of the occupational 
programs. For the secondary programs, we found but one switching of positions 
of two adjacent programs. 0:i the whole, the ranking of the programs showed 
stability ever the 3-year period 1966- 1969. 



Second, a sample of State capital cities was!; visited to discuss data 
gatherings procedures with vocational education officials and learn their 
assessment of the validity of the reports. From those! visits, it was con- 
cluded that although there were problems and uncertainties, the reports 
generally reflected conditions with reasonable accuracy. 

Third, the data were subjected to certain statistical tests* The 
analysis of variance described above also has implications for assessing 
the validity of the reported data. If the reports had bpen invented or fic- 
titious, it is likely that all of the occupational groups', would have tested as 
having coma from the same population; according to tpe variance test, how- 
ever, the data for the different groups came from different populations. 

Another aspect of the data reflecting on their validity is the frequency 
distribution of the values. Frequency Jistributions,;were developed for each 
occupational program, and in most cases the ratios within occupations pro- 
grams follow a distribution that is unimodal and positively skewed, which 
shculd be expected for this type of activity. An inference of homogeneity with- 
in groups can be drawn from such distributions. This inference could not be 
drawn for only two of seven programs tested, and in both of these cases the 
number of obseivations was relatively small. On the whole, therefore, the 
frequency distributions of the observations indicate homogeneity within groups, 
which would be expected from valid data. 

Inferences from frequency distributions and statistical testing, when 
taken individually, cannot be regarded as sufficient indicators of the accuracy 
of the data reported. When taken together, however, «,nd while considering the 
highly dispersed sources of the reports, they are highly indicative: the dif- 
ferences among programs are significant and the distributions within programs 
show homogeneity; thus, considering that the data originated in many different 
jurisdictions, it would be highly improbable for the reports not to reflect actual 
conditions. In sum, although the evidence of course is not conclusive, there 
is no evidence that the data are unreasonable and then fore unusable for analysis. 
The statistical features usually associated with inaccurate reports are not char- 
acteristic of the data reported through the annual report on vocational education. 

IMPLICATION OF USING TOTAL COSTS PER ENROLLEE RATHER THAN PROGRAM 
COSTS PER ENROLLEE 

As was mentioned earlier, the vocational education expenditures used 
to develop cost-effectiveness ratios are program financial costs, not total costs 
per student. 1/ The costs incurred per student are substantially higher than pro- 
gram financial costs, which reflect only the direct instructional costs for 
vocational education. Construction costs are mostly excluded, and when voc- 
ational education is taught in comprehensive school i any of the joint costs 
are not reported as vocational. Also, the costs of no: v ational courses taken 



Foregone earnings and similar costs that might be regarded as part of the total 
costs are not considered here. 




38 



4S 



by vocational students are omitted from expenditures reported for vocational 
education. To illustrate the extent to which this can be a factor, 12 percent 
of secondary vocational education students take only one course in vocational 
education. An equal portion of students take only three courses. Only about 
half of the vocational education students cany five or more vocational courses; 
thus there is considerable difference if only instruction is considered, with- 
out regard to the number of courses taken. 

One report on costs presents the following data comparing average per 
student costs for two types of post -secondary training:-?/ 

Average 

ins tructional costs Full c osts 

Community colleges . . . $599 $1,184 

Technical schools .... 844 3,664 

Per student costs for secondary vocational education are shown in a number of 
reports using data from the mid-sixties. The estimates range from $500 per 
year to $95 0 per year— not including capital expenses, which appear to repre- 
sent from 8 percent to 20 percent of instructional costs. 

A comparison of secondary and post-secondary program costs by occu- 
pational category, as calculated in this way from the annual vocational educa- 
tion reports, appears in Table 9. 



TABLE 9.— Program financial costs per enrollee, FY 1967 



Program 


, 

Sccondaiy 


Post-secondary 


Agriculture 


$ 1 5 1 


$587 


Trades and industry . . 


261 


347 


Technical 


33 3 


412 


Office 


69 


128 


Health 


149 


418 


Distributive 


2 14 


210 


Unweighted avg . . 


200 


350 



Source: USOL, Vocational and Technical Education Annual Report — 
fiscal Year 1967, U.S. Govern lent Printing Office, Washington, D. C . 



■^William G. Morsch, Study of Commu n ity Colleges and Vocational Trainin g 
C enters: Cos t Analysis (unpublished report prepared for the Department of 
Health, Education and Welfare, USOF, Bureau of Social Science Research, 
Inc., Washington, D.C., 1970,) 
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Comparing these figures with those appearing in the reports mentioned 
above! program financial costs can be seen to be considerably less than total 
costs per student. In the following tabulated comparisons! unweighted averages 
and mid -points are used for the data that were available: 

Annual program financial Annual total costs 
costs per enrollee per enrollee 

Secondary .... $200 $ 725 

Post-secondary . . . 350 1,424 

These figures show that total per student costs for secondary vocational educa- 
tion are about 3.6 times the program financial costs, while total per student 
cosis at the post-secondary level are 4 times the program financial costs. 

The cost-effectiveness ratios presented earlier, being based on pro- 
gram financial costs, are suited to showing the relative efficiency of expendi- 
tures on the different occupational programs in terms of securing jobs for gradu- 
ates. In terms of absolute dollars incurred per student, however, they are greatly 
understated. An approximation of the total per student costs for those actually 
placed can be made by scaling up the ratios by the factors of 3.6 and 4 for 
secondary and post- second ary, respectively. Such an approximation would be 
rough because, as mentioned earlier, the scaling factors are averages with some 
of the values derived from sparse samples; also, these factors do not reflect 
any differences by occupational program (i.e. , they cannot discriminate among 
the different occupational categories). They can, nevertheless, be applied to 
the cost-effectiveness ratios that were developed for each occupational program, 
to produce ratios that more accurately reflect the total per student costs incurred 
by each. The scaled ratios are given in Table 10. 

It can be seen from these figures that the cost of vocational education 
is substantial when all of it is attributed to the graduates who actually obtain 
jobs in their field of training. Here again, the variation among programs is 
considerable. At the secondary level, it costs three times as much to place a 
graduate in a technical or an agricultural position as it does to place one in an 
office occupation. At the post-secondary level, it costs over four times as much 
to place a graduate in agriculture as in a health occupation. 

The reader should be warned to use extreme caution with regard to these 
estimates , They are based on fragmentary data and wer? developed only to per- 
mit the compilation of very rough estimates of the full cost implications of the 
placement of an individual in a job related to his training. "Loading" the indi- 
viduals so placed with full program cost carries the implication that those not 
in related jobs received no benefits from their training , with the consequence 
that none of the cost is assigned to them. In addition, the scaling factor had to 
be derived from fragmentary data— the only data available to this analysis —rep- 
resenting a single locality, although it s well known that there is considerable 
variance in costs among localities. Since the scaling factor is used as a multi- 
plier, moderate differences in it would result in larger c if ferences in the product 
ratios . 






t In assessing the results oi these calculations, it should also be kept 
in mind that the costs do not represent single-year costs. Dividing the year's 
program costs (whicn encompass 4 years) by the number of placements in related 
jobs has the effect of including all of the years of the graduates* education in 
the costs. Using full program costs also has t h e effect of "charging the gradu- 
ates' 1 the costs incurred by dropouts, even dropouts who have in fact gotten 
related jobs. This was not a matter of choice; follow-up data on dropouts are 
not available. It should be noted, further, that no provision has been made 
for tXe sizable portion of graduates about whom nothing is known. Computa- 
tionally, they are all treated as not having obtained jobs related to their edu- 
cation, and their costs are shifted to the graduates that are known to have been 
placed in such jobs. In sum, the values should be treated as rough approxima- 
tions having a number of areas of uncertainty in their derivation. 

An important conclusion from this analysis is that there is a need to 
improve the analytical data base so that a more precise assessment can be 
made of the quality of the graduates' occupational training experiences. An 
examination of nonplacement benefits should also be possible, as should some 
appraisal of the displacement effect (which, for example, may be less in the 
technical occupations than in the distributive), although on the basis of cost 
per placement the technical training appeers less effective. 



TABLE 3 0. —Cost-effectiveness ratios increased to reflect total costs 
per student placement in a training-related job 



program 


1 Secondary 


Post-secondary 


Office 


$ 3,124 


$ 7,624 


Distributive . 


3,330 


8,076 


Health .... 


4,766 


4,680 


Home econ 






(gainful) . 


5,519 


7,664 


Trades and 






industry 


7,038 


10,368 


Technical 


9,821 


16,832 


Agriculture . 


10,415 


20,244 
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Thus far we have been attributing costs, whether they be program finan- 
cial costs or full costs per student, to those who have been placed in jobs 
related to their training. The estimated costs per placement are high, for two 
basic reasons. First, per student costs for vocational education are relatively 
high, and second, only a portion of the graduates obtain jobs in fields related 
to their vocational education. Estimates can be made of the amount of ’unpro- 
ductive" money that is spent on these students (i.e. , unproductive in achieving 
placements related to student training) ,-2/ as shown in Table 11. 

Calculations were presented earlier of the percentage of graduates that 
obtained jobs in training-related occupations. The balance of the students were 
either unemployed, employed only part-time, employed in jobs unrelated to their 
training, in the Armed Forces, involved in additional full-time schooling, or 
could not be found. A statistical proration of costs can be attributed to the 
balance of studenls who received training but did not use it in an occupational 
context. The statistical proration was made through use of the following: 

51 
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i=l 

where 

P - percentages of completers obtaining jobs ir» 
related fields 

E = expenditures made in the State. 

There are major data needs to improve the type of strategy evaluation. First, 
the classification scheme of occupational, or program, categories should be 
improved. The classification system is too aggregative. More and better classi- 
fications are required to clearly delineate categories such as "Technical 11 and 
"Trades and Industry." Another major shortcoming is in the time horizon of 
follow-up, which extends only to the October following graduation. A 5 -year 
follow-up would contribute significantly to program evaluation. 

In general, this analysis can be useful in providing an overview of 
the relative performance of the different kinds of training. It cannot, however, 
bo used to explain why the programs performed as they did, or how the programs 
performed on a greater set of needs. 



The term "unproductive" is not to imply that graduates who do not get train- 
ing-related jobs do not benefit from their education in other ways . 
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TABLE 1 1 , —Estimates of vocational education expenditures made on students 
who did not obtain jobs related to fields of training or who were not found 





Secondary 


Post-secondary 


Program 


% not 
obtaining 
related 
jobs-!/ 


Estimated 
cost of 

training 


% not 
obtaining 
related 
jobs-!/ 


Estimated 
cost of 
training 


Distribution . . . 


58 


$ 1 8 , 930 ,482 


56 


$ 3,404,656 


Office 


53 


66,710,940 


35 


15,405,353 


Health 


66 


6,342,086 


25 


8,628,708 


Trades and 

industry . . . 


57 


76,095,111 


43 


227,955,174 


Agriculture ■ ■ • 


68 


58-380,835 


42 


3,275,012 


Technical . . . 


75 


8,259 ,692 


43 


22,746,553 


Home econ . . * 


74 


5,265,743 


57 


1,161,618 


Overall . . . 


60 


$239,992,888 


39 


82,577,074 


Total program 
expenditures 




$405,083,223 




$213,040,678 



1 / 

These values are arithmetic means and are not comparable to the median 
values shown in Table 4 . 
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APPENDIX C 

VOCATIONAL EDUCATION PLANNING 
AND EVALUATION ACTIVITY 



INTRODUCTION 

This appendix provides an overview of planning and evaluation activity 
within the realm of Federally-funded vocational education. It focuses on the 
activities of groups responsible for planning and evaluation at the Federal, 
regional, State, and local levels. 

The basis for this presentation is information obtained in part through 
discussions with vocational education personnel involved, in various capacities, 
with planning and evaluation. Other information was obtained from various voca- 
tional education reports, working documents, and plans that indicated the man- 
ner in which planning and evaluation is being conducted at the various levels. 

The bulk of all information utilized was obtained at the Federal level. 

Summary 

The planning and evaluation of Federally-funded vocational education 
programs may be viewed as a continuum of activity from the Federal level, 
through the regions and States, to the local level. It may be viewed as an inter- 
active process involving each level with the next above and/or below. 

Planning and evaluation at each of the levels is affected both by guide- 
lines and constraints imposed from above and by limitations imposed from belcw. 
For example, at the Federal level national vocational education priorities and 
budget appropriations generally are determined at the department and office levels; 
planners and evaluators will work within these, having little control over them. 

At the same time the planners and evaluators are being called upon for data on 
'current programs, projec 'o^s of future requirements, and evaluative information 
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on what works and does not work in vocational education, and the data they 
require to respond in these areas are in general not available from the States 
reporting to them. 

A similar situation exists at the State level. Planners and evaluators 
must be responsive to both Federal and State legislation and priorities, and 
they must work within the imposed budget limitations. They often are expected 
to respond to data requirements of the Federal level and do not, in general, 
have the mechanisms to obtain the needed data from the local education agencies, 
although a number of States are developing this capability. Because of the 
autonomous nature of public education, there is little leverage from Federal to 
State, or from State to local area. 

At the Federal level vocational education planners and evaluators devote 
time to responding to the requiiements of the overall Government planning cycle. 
They also provide assistance and support to the regional, State, and local vo- 
cational education operations, in an effort to improve the capability to plan and 
evaluate, and thus improve the quality and scope of information provided to the 
Federal agencies. Assistance and support are also given to the national and 
State advisory councils for vocational education. 

The rcle of the regional level is emerging now in the area of vocational 
education planning and evaluation. Consistent with the recent Federal move- 
ment toward regionalization of program administration, regional vocational 
education personnel are assuming a larger role in planning and evaluation. 

The regions are assisting Federal headquarters by taking responsibility for the 
review and approval of State plans. They are also developing the capability 
to assist their States in the planning and evaluation of State-level programs. 

In the States, planners and evaluators are responsive to requirements 
from above, but they also devote time to working with local education agencies 
to develop improved local capability and improved data for use at both local 
and State levels and for use in reporting to the regions and Federal headquarters. 
The State plan appears to be a model for planning and evaluation activity at the 
State level. 

local planning probably shows greater variation in style, format, 
and comprehensiveness than planning at any other level. The scarcity and 
inconsistency of data, combined with whatever restrictions are imposed at the 
local level (legal, political, or otherwise), apparently affect the quality of 
local plans. A few more progressive districts are moving toward a learning 
systems approach that will be based on program planning and budgeting (PPB) 
techniques and will place heavy emphasis on evaluation. Others are attempting 
to improve outreach and increase the effectiveness of existing facilities by 
joint cooperative planning with adjacent areas. 
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While planning and evaluation personnel at all levels are stressino 
the development and improvement of information systems that would provide 
usable data for measuring program effectiveness, current systems are not yet 
providing a data base that would support a comprehensive planning or evaluation 
program. At all levels, independent research efforts are relied upon at this time 
for most needed information . Federal planning and evaluation personnel are also 
stressing the need to develop statements of measurable objectives for vocational 
education at all levels, beginning with the Federal. 

Finally, it is apparent that program evaluation at all levels is severely 
hampered by the lack of an agreed upon measure of effectiveness (e.g., cost per 
student placement). Far too much reliance has been placed on qualitative 
indices of effectiveness in the absence of more precise, quantitative measures. 

FEDERAL PLANNING AND EVALUATION 

At the Federal level, the responsibility for vocational education pro- 
gram planning and evaluation is lodged with the Planning and Evaluation (P&E) 
Branch in the Division of Vocational and Technical Education, Bureau of Adult 
Vocational and Technical Education (DVTE, BAVTE), This branch is organized 
into three sections — Planning , Evaluation, and Analysis and Reposing. The 
three sections are staffed by 10 professionals in addition to the branch chief. 

In examining the functions and activities of the Planning and Evaluation 
Branch, it may be helpful to group them into two major areas, the first dealing 
with planning and evaluation support provided within the Federal structure (to 
the U.S. Office of Education and HEW), and the second dealing with planning 
and evaluation support provided to other levels — the regions, the States, and the 
local education agencies (LEAs). 

Support Within Federal Structure 

The planning and evaluation activities of the P&E Branch within the 
Federal structure may be characterized as essentially responsive to needs 
identified at higher levels within HEW ard USOE. 1 he annual HEW planning 
cycle provides the basis for all planning activity relating to vocational educa- 
tion at the Federal level. (See "FY 1972 Planning Calendar” included at the end 
of this appendix.) Within a framework of annual goals and objectives determined 
for HEW at the Secretary's level, the Commissioner of Education identifies pre- 
liminary issues and program thrusts, and requests responses and comments on 
these from the bureau level. The "issue papers' 1 that are written in response 
to these requests are the primary means of inputting planning and evaluation 
information upward through USOE. The issue papers are an attempt to assemble 
as much information as is available to the P&E Branch to describe the problem 
or situation in question and influence the programming needed to deal with that 
problem or situation. 
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A second activity carried out by the P&E Branch as part of the annual 
planning cycle involves the provision of justification for the budget lines or 
budget estimates for vocational education programs each year. The P&E Branch 
has no role in the determination of the funding levels for vocational education, 
in that these are predetermined for all Federal offices and programs at the de- 
partment level. Rather, the P&E Branch responds to the budget estimates given 
with information on need for the programs (and funds) and plans for the expenditure 
of the funds by the programs. Here again, ail relevant information available to 
the P&E Branch is utilized in providing the justification for the budget line. 

In addition to responding to issue papers and developing budget 
justifications, the P&E Branch is also called upon to review and comment on 
USOF/s "Program Memorandum/’ prepared by the Office of Program Planning 
and Evaluation (OPPE) under the deputy assistant secretary for monitoring 
and evaluation. Since this memorandum is the complete statement of the USOE 
programs proposed for the following fiscal year (it is based upon issue papers 
or selections from them), the P&E Branch reviews it to ensure that vocational 
education priorities and program plans are properly described and clearly repre- 
sented. The responses made by the Po*E Branch are reviewed, and may be 
modified, at the bureau level, before submission to OPPE. 

The P&E Branch is attempting to coordinate all of its activities with the 
recently introduced "management by objectives" (MBO) plan of HEW. Under MBO , 
in general, the Secretary states priority goals for the department for the fiscal 
year, the Commissioner of Education translates these into objectives for educa- 
tion, b'lreau chiefs develop appropriate objectives and action steps, and division 
directors develop the activities required to ultimately contribute to objective 
attainment. The P&E Branch develops the specific planning and evaluation 
activities it will carry out during the year in response to the MBO. However, 
it should be noted. that there are few policies and procedures covering MBO at 
this time; it has not yet been completely reconciled or integrated with the 
existing annual planning cycle, and whether budget makers pay a great deal of 
attention to MBO has yet to be determined. 

Support to Othe^ Levels 

The second major area of planning and evaluation activity at the Federal 
level-support to the regions, States, and local education agencies-may be 
subdivided into the three following areas: 

P&E Branch staff organize, conduct, coordinate, or 
simply attend conferences and seminars on planning 
and evaluation at all levels. For example, since 
1967 seminars have been held to introduce PPB con- 
cepts to vocational education planners at the State 
level. Staff members also devote time to delivery 
of speeches at various functions concerned with 
vocational education planning and evaluation. 
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b. P&E Branch staff provide technical assistance at 
all levels, including staff training in regional 
offices, work with data reporting personnel in 
State offices, and work with State advisory councils 
and administrative and program officers at ail 
levels on planning and evaluation for general 
administration and for specific content areas. 

c. P<&E Branch staff publish both statistical reports 
and working documents for use at all levels. 

In addition to the Technical and Vocational 
Education Annual Report , which summarizes and 
analyzes the statistical data contained in each 
of the State* s Annual Vocat i onal Education Re - 
port (six issues thus far), which is used for 
its conferences and seminars. Staff members 
also provide articles on planning and evaluation 
to journals and magazines . 

Comment , The influence that the P& E Branch can exert on planning 
and evaluation below the Federal level is constrained by a number of factors. 

Of the funds allotted for vocational education programming, 9 0 percent are 
granted directly to States based upon State population by age groups needing 
vocational education and per-capita income. Further, the Federal funds 
inputted represent a very small portion, when compared to State and local in- 
puts, of the total expenditure for vocational education programs. Administra- 
tion of the funds is the responsibility of the State education agency (SEA); and 
the local education agencies are responsible fci the conduct of the programs. 

Because of the autonomy of each S ^ and LEA, Federal influence is 
necessarily restric Led to the aforementioned ictivities centering around con- 
ferences and seminars, technical assistance, and publications. It should be 
noted, however, that according to P&E Branch staff members, State and local 
vocational education personnel have been very receptive to this Federal support 
and very responsive to its content. They r eport thM. when new ideas or approaches 
to planning and evaluation are imparted to '"'ate and local personnel, they are 
very often visible in subsequent planning anu evaluation acMvity within those 
SEAs and LEAs . 

Data for Planning and Evaluation 

The most serious constraint affecting planning and evaluation at all 
levels is the lack of sufficient baseline data on program operations. According 
to P&E Branch staff members, no funds have been made available within voca- 
tional education for the development of the "benchmark 1 ' data that would be re- 
quired for effective planning or evaluation. 



O 



AO 



The data that are now relied upon by the ?& E Branch for use in all of 
its activities and publications are drawn from (a) the State pLans and annual 
reports submitted to the Division of Vocational and Technical Education by each 
State; (b) Department of Labor reports and publications containing manpower and 
labor statistics; (c) published studies from professional journals and other sources. 
In addition to these sources, the P& E Branch relies heavily on the individual 
evaluation studies for which it contracts with various public and private research 
organizations. For FY 1970 the entire budget line of $900,000 for planning and 
evaluation (both share the same budget line within USOE) was committed to nine 
major contracted evaluation projects and some miscellaneous small projects and 
support activities in evaluation and monitoring (see list of education and evaluation 
projects included at the end of this appendix). P&E Branch personnel feel that 
in the absence of any funding for the development of an information system for 
vocational education program monitoring at the S+ate and local levels, the con- 
tracted evaluation studies will continue to be relied upon heavily as the basis 
for planning priorities and recommendations at the Federal level. 

The time lag in the information flow that now exists is also a problem 
affecting planning. During calendar year 1970 the annual HEW planning cycle 
for FY 1972 is underlay. At the same time, the program data obtained through 
the annual reports from each State are still being compiled for the FY 1969 
Vocational and Technical Education Annual Report. 

Because of this general lack of baseline data on vocational education 
programs, the P&E Branch is limited in the planning and evaluation methodology 
it may employ. In the area of planning, treatment of data consists mainly of 
analysis of trends in program enrollments, analysis of manpower and labor mar- 
ket data for trends in occupational areas, analysis of results of contracted re- 
search, and review of advisory committee recommendations and expert opinion 
regarding future vocational education needs— all of which contribute to projec- 
tions of program requirements by area, level, type, for use in issue papers, 
budget justifications, reports, etc. 

In the area of evaluation, the contracted evaluation studies are relied 
upon completely for the assessment of vocational education program operations. 
Evaluation personnel are concentrating on providing assistance to SEAs and 
LEAs— helping them to develop the capability to identify measurable objectives, 
and to develop and assemble the baseline data required for identification of 
objectives and subsequent measurement of attainment. Evaluation personnel are 
preparing a manual containing guidelines for Federal interaction with States for 
program evaluation. Further, studies are now being developed by the P&E 
Branch that will begin picking up program effectiveness data in addition to the 
input-type data that are now collected. The P& E staff anticipates that with 
HEW support (resources), these studies could be expanded during the next 2 to 
3 years. They are currently working with top training managers of companies 
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such as General Motors, Ford, and Xerox, who are apparently very supportive 
in this effort to develop effectiveness studies. 

Impact of Vocational Education Amendments 

The Vocational Education Amendments of 1968 should have a positive 
effect on planning and evaluation at all levels. They spell out the role of the 
National Advisory Council on Vocational Education to include review of the 
operation of programs, and evaluation of their effectiveness , The amendments 
also spell out the role of the State advisory councils to incLude evaluation of 
programs, services, and activities, as well as the publication of an annual report 
containing evaluation results aid change recommendations. Under Part B, Section 
122, funds are available to States to assist in development of the required State 
plans, administration of the plans and collection of manpower and labor market 
data, and evaluations of programs and dissemination of results. A most important 
requirement included in the amendments under "State Plans" is that local educa- 
tion agencies shall submit applications for vocational education funds to the 
State level. This means the LEAs must involve advisory groups in planning, must 
ensure that iocal plans display local capability to meet the needs of the target 
population and labor market, and must provide for periodic evaluation of the pro- 
grams in operation. 

The thrust of the amendments is toward process- oriented planning, at 
the Federal, regional, and State levels, that communicates priorities and 
channels Federal funds to the right programs. P<5 E Branch personnel feel that 
the focus of the 19 68 amendments on improved planning shouLd enhance the 
accuracy and utility of the State plans and the program data that will be reported 
from the local to the State to the Federal level. 

R ole of OPP E 

While the Planning and Evaluation Branch is focusing exclusively on 
vocational education programming, the Office of Program Planning and Evaluation, 
under the deputy assistant secretary for Planning Recearchand Evaluation, is 
responsible for support in planning and evaluation throughout the Office of 
Education. OPPE consists of three divisions, with the Post-Secondary and 
Special Education Programs Division having the primary interface with the ?& E 
Branch for work in the area of vocational education. OPPE is heavily involved 
in the annual HEW planning cycle, coordinating the preparation of program 
memoranda, program financial plans, and issue papers. 

In the area of planning, OPPE strives to make key issues in education 
(including vocational education) highly visible within USOE and HEW. OPPE 
reviews the research studies that are performed, summarizes information relevant 
for policy-making and submits it to the USOF Policy Advisory Board which, at 
this point in time, is functioning only as a forum for discussion of key issues 
in education. 
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A major activity involving both OPPE and the P<& E Branch deals with 
the contracts let tor evaluation studies in vocational education. P£<E Branch 
personnel devote much time to the development of the requests for proposal 
(RFPs) for the contracts. The P&E Branch develops the research tasks for which 
studies would be required; OPPE reviews and approves these tasks and assigns 
the funding level for them; the P&E Branch writes the work statements for the 
RFPs; both are involved in review of proposals, selection of contractors, 
and monitoring of studies during the contract period. 

In the area of evaluation, a major responsibility of OPPE is the de- 
velopment of the USOE’s 11 Annual Evaluation Plan," This plan structures and 
defines all of the evaluation studies to be sponsored by OPPE as well as all of 
the individual USOE bureaus, and in so doing takes into account such factors as 
HEW-OE priorities, legislative mandates, Executive Office interest, public 
interest, etc. 

Role of the National Advisory Gou nciL 

The National Advisory Council ^n Vocational Education supports the 
planning and evaluation activities conducted within USOE in a number of ways. 
Its 21 members are appointed by the President. Currently the council includes 
representatives of the educational community, industry, post- secondary voca- 
tional training programs, the disadvantaged , and minority groups. In addition, 
a student was appointed to the Council for the first time, in January 1971 . 

(See Volume I, Chapter V,of this report for more detailed discussion of Council 
membership.) It employs a full-time executive director and support staff By 
virtue of its members' broad representation of interests and their individual 
expertise in the area of occupational education, the National Council is 
equipped to advise USOE on matters of program regulation, administration , and 
operation. It reviews vocational education program effectiveness, makes recom- 
mendations for modifications, and submits annual reports to the Secretary of 
HEW covering its findings and recommenda tions s The Council may contract with 
private institutions or organizations for program evaluations. 

REGIONAL PLANNING AND EVALUATION 

The regional role in planning and evaluation is currently changing . 

As recently as 1965 most regional program officers were oriented toward man- 
power and curriculum areas, for example, contract officers for MDTA and, 
somewhat later, subject matter specialists. In fact, until FY 1 970 there was 
little, if any, planning and evaluation performed at the regional leveL. Impetus 
for increased regional participation has evolved from the requirement that each 
State submit an annual and long-range State plan for vocational education. 

The regional directors of vocational education and their staffs have 
been providing guiiance to the States in the development of thu State plans. 

In reviewing individual plans they have attempted to help underscore important 



areas,' such as the specific needs of the peopLe residing in the State and priorities 
which should be attached to training in specific occupational categories. From 
a broader standpoint the regional office has been concerned with what might be 
called "the process of planning" employed by the States, i * e . : How do the 
individual States intend to implement programs so that the results will be both 
achievable and measurable? Thus far, in the absence of solid data, the regions 
have had to rely heavily on information generated within the State itself and made 
available through the cooperation of the State employment service and the Bureau 
of Labor Statistics. During the current fiscal year State plans will not be forwarded 
to Washington, D . C . , as they were in the past; instead , the entire review is to 
take place at the regional Jevel # If the individual plans are approved, a memo- 
randum to that effect originating from the regional director of vocational education 
will be all that is required to certify eligibility for TederaL funds. (WhiLe not 
required for approval purposes, copies of State plans may in fact continue to be 
sent to Washington for information purposes.) 

In FY 1972 , although major emphasis wilL still be on the "planning 
process," the regional office will continue to expand its role vis-a-vis the 
States within its jurisdiction. In accordance with the general philosophy of 
"management by objectives," the objectives of the Secretary of HEW and the 
goals established by the Commissioner of Education will be translated into 
priorities by the regional director. The regional office wilL meet with State 
representatives and seek to identify measurable objectives that coincide with 
regional priorities. An integral part of this process is the development of an 
operational planning document which, while currently in an embryonic state, 
holds great promise for the future. High on the list of objectives is the complete 
changeover within each State to a PPB system. 

Data for Planning a _ nd Evaluation 

There are two primary sources of data available for evaluation at the 
regional level: the annual report submitted by each State, and "on-site" visits. 

The latter are initiated at the regional leveL and usually involve a team of 
four or five program officers who go to a given State with a prepared agenda. 

The questions generally asked include (a) What programs are underway? 

(b) How are the programs organized? (c) How does the State make decisions ? 

(d) Why were certain decisions made? (e) Have target groups been reached? 

On-site visits occasionally are made in local districts to see how welL the State 
plan is being implemented. 

At least one region is planning to expand its role during the cominy 
year to include a special evaluation of two major occupational areas. Although 
these areas have not been designated as yet, there is e strong possibility that 
construction trades and health occupations will be selected. There are also 
indications that some attempt will be made to develop evaluation instruments 
that can be used in all the States within the same region. 
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STATE PLANNING AND EVALUATION 



The Vocational Education Amendments of 1968 stipulate that "any state 
desiring to receive the amount for which it is eligible for any fiscal year 
pursuant to this title shall submit a State Plan . , . which meets the require- 
ments set forth in this title," The State plan itself is organized into three 
parts. The first part contains the administrative provisions that will govern 
the expenditure of the vocational education funds in the State and assurances 
that all requirements of the act will be met— with explanations of how they 
will be met. The second part contains the provisions for the proposed long- 
range (5-year) program for vocational education in the State, including narrative 
and statistical information describing current and projected programs and re- 
quirements. The third part contains provisions for the proposed annual program 
plan, including descriptions of all of the programs to be provided, numbers of 
programs, teachers, students, graduates, and related information. Because 
the State plan is (a) a required contractual agreement, (b) a description of what 
will be needed and offered each year, and (c) a 5-year projection of future needs 
and activities, it represents the focal point for all planning and evaluation at 
the State level. This is the first time that States have been required to report 
on their effectiveness in reaching theit state objectives. 

Each state board of education has ultimate responsibility for the 
administration of the plan. Thus, the board ultimately is responsible for State 
plcinning and evaluation; specific planning and evaluation activities, however, 
are performed by various groups and individuals within, or associated with, the 
State departments of vocational education. The people involved at the State 
level would, in general, fall into three groups: 

a. State advisory councils w ork in an advisory capacity 
with the boards of education. Councils review State 
plans and contribute their experience and expertise 
to establishment of objectives and priorities for 
programming in the State. 

b . Bureaus /divisions/programs within the departments of 
vocational education have specific responsibility for 
State planning and evaluation across ali vocational 
education program are^s. Such groups may include 
research, survey, exemplary programs; bureaus of 
program planning and development; bureaus of pro- 
gram services and evaluation; and service area super- 
visory units. The functions of planning and evalua- 
tion may or may not be nandled together by a single 
group in any given State. The research coordinat- 
ing units (RCUs) in many States are heavily involved 
in research and evaluation activity, and to a lesser 
degree in planning. Although the activities of RCUs 



vary from State to State, generally speaking these 
units are responsible for development, coordination, 
and assistance in performance of research and 
evaluation for program improvement and planning 
purposes . 

c . Specific program areas within the departments of 
vocational ed ucation are responsible for planning 
and evaluation of their own programs. Included 
here are organizational units that advise agriculture, 
distributive education, home economics, and other 
programs, that are responsible for determining pro- 
gram objectives and priorities for State planning 
purposes, and that conduct evaluations of the 
ongoing programs in their specific areas. 

Program Planning- Long Range 

At the State level, planning may be characterized as an interactive 
process involving each of the three basic groups just described. The following 
planning steps are structured in the State plans and will involve all of the 
groups to some extent, with the second group having perhaps the heaviest in- 
volvement, since it is responsible, in most States, for the coordination and 
completion of the overall long-range planning activity each year. 

Just as vocational education planners at the Federal level must be 
responsive to program priorities and budget limitations determined at higher 
levels in the government, so too must State planners be responsive to the 
dictates of legislation, Federally-determined priorities, and limitations of funds 
allocations to the States. Given these fundamental guidelines and constraints, 
the basic planning steps performed include the following: 

a. Analyze manpower needs and job opportunities in 
the State. These must be determined to provide the 
basis for vocational education programming that 
will ensure a match between programs and the labor 
market. 

b. Analyze availability of vocational education in the 
State. This involves identification of specific 
economically depressed areas, areas in which there 

is a high unemployment rate, areas in which there is a 
high school dropout rate, and the areas of greatest 
population density. 

c. Analyze the State's population in terms of vocational 
education needs. This involves analysis of the 
characteristics of the population, both current and 
projected. Characteristics relevant to vocational 
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education planning include population oy secondary 
school, post-secondary, and adult age; physical 
and mental handicaps; socioeconomic status, etc. 

d. Specify vocational education program needs. This 
involves the examination of the three foregoing areas 
of analysis to determine the priorities for vocational 
education programming in the State. Priorities are 
stated in terms of target groups and target areas to 
be served, and in terms of employment needs and 
labor demand. 

e. Specify vocational education obj ectives. In addition 
to and consistent with program need priorities, voca- 
tional education objectives are stated in terms of 
levels, program areas, and numbers and types of 
students who will be enrolled and who will com- 
plete programs 1 and 5 years into the future. For 
example, enrollment objectives will be spelled 

out for secondary, post-secondary, and adult; 
disadvantaged and handicapped; eor.sumer and home- 
making; cooperative and work study. 

f. Analyze State's vocational education program. This 
includes a breakdown of program enrollment, numbers 
and types of vocational education schools, construc- 
tion projects, numbers of teachers and teacher train- 
ing enrollment, and estimates of total funds needed 
for programs as planned for the following 5 years. 

To conclude the review of State level vocational education planning 
and evaluation, two significant points should be made. First, the lack of 
accuracy and precision in existing manpower surveys continues to be a major 
barrier to effective vocational education planning. Second, in addition to items 
such as population characteristics, unemployment, job openings, and employ- 
ment forecasts, other factors— including training costs and wages— deserve 
greater attention in the planning process at the State level. 

Research for Planning and Evaluation 

Jn addition to planning oriented to operational programs in vocational 
education, planning priorities and objectives are also developed for areas of 
exemplary and research programs . Selected examples of objectives ior exemplary 
programs include 

• Increase or develop community and Industry 
involvement and coordination with educators 
in providing improved vocational education 
offerings . 
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• Provide a work orientation program for 75 percent 
of students at K-6 grade level that will encourage 
constructive work attitudes in all youth. 

a Institute series of workshops to be used in planning, 
implementing, and evaluating vocational education 
programs . 

Selected examples of objectives for research programs include: 

• Develop procedures for evaluating projects for the 
disadvantaged and handicapped. 

• Evaluate labor force demands compared to voca- 
tional training supply. 

• Develop occupational trends forecasting information 
systems 

• Institute computer-assisted vocational education 
instruction . 



The exemplary and research programs at the State level are concerned 
with improving vocational education programs and their administration, but 
also represent a significant portion of the program evaluation that is performed 
or is planned to be performed over the projected 5-year period. 



Program Planning— Annual 

While the long-range planning activity focuses on the establishment 
of program priorities, objectives, and projected service needs and levels, the 
annual planning activity takes each program area (agriculture, distribution, 
health, etc.) down to the instructional leve l— descri bing the levels at which 
each program will be offered, the number of programs to be continued or 
expanded, the number of teachers, and the estimated enrollments and com- 
pletions for each program for the year. 

Based upon all of the analysis performed as part of long-range planning, 
the annual plan determines for each program area, the target area, the target 
groups to be served, the geographic areas to be served, the occupational offerings, 
as well as facilities construction plans, teacher education plans, curriculum 
development plans, etc. There is some question at the Federal level, however, 
whether the annual State planning activity involves sufficient scrutiny of 
alternative approaches to programming and alternative methods of delivery. It 
is felt that State planning capability in general has not yet developed to this 
point . 



Annual planning activity also involves each of the three groups 
previously described. Again, the second group- the bureaus , divisions, and 
programs within State departments of vocational education — is the most heavily 
involved in coordinating the inputs of the other two groups, assembling the 
data, and producing the plan. 
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Evaluation 



The long- range plans for each State describe the research and exemplary 
programs, which include the planned program evaluation projects. The annual . 
plans describe how evaluation will be performed, and by whom, for each year.-' 
In general, responsibility for evaluation is shared by the three groups previously 
described . 

S tate Advisory Councils. Each council is now required to submit as 
part of the State plan an evaluation report on the effectiveness of vocational 
education programs and services in meeting the annual and long-range program 
objectives. This report, requested by the Vocational Education Amendments 
of 1968, is now being submitted to the Federal level for the first time, and so 
its contribution to the overall evaluation effort has not yet been assessed. In 
some cases, councils may employ full-time staff members responsible for pro- 
gram evaluation; in other cases, councils may contract with institutions (univer- 
sities) or private research organizations for evaluation plans or studies. 

State Vocational Education Units. Specific organization units within 
the State departments of vocational education are responsible for evaluation 
of programs overall. In some cases, this is simply a check on local programs to 
ensure they are in compliance with Federal and State regulations. In other 
cases, it involves working with local education agency personnel in developing 
their capability to conduct program evaluation. As previously mentioned, the 
research coordinating units in some States play a central role in planning and 
evaluation. Examples of RCU functions include: 

• Development of State evaluation models, including 
systems for collection, storage, retrieval of data 

• Identification of needs and assignment of priorities 
for research, evaluation, program development, 
teacher training programs 

• Preparation of special studies (e.g., to support 
formation of vocational education school districts) 

• Preparation of annual vocational education reports 
for the State 

• Preparation of both annual and long-range program 
provisions ioi the State plan 



Some critics contend that the majority of State plans are of little use for 
the purpose of sophisticated evaluation. In general, badly needed informa- 
is either completely lacking (e.g. , data on student aptitudes, follow-up, 
etc.) or is too gross to permit proper analysis. 
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• Collection, summarization, dissemination of research 
findings relevant to vocational education. 

In regard to program evaluation, the RCU also may bo responsible 
for evaluation studies that are contracted by the State to institutions or 
private research organizations. It is felt at the Federal level that although 
the RCUs are well equipped and have the technical ability to work effectively in 
planning and evaluation, they are probably not being fully utilized by all States 
in these areas. 

At the State level as well a; the Federal l^vel, evaluators must rely 
heavily on contracted research. While a number of States are developing 
overall evaluation plans and data collection systems to support them, few 
State level evaluation systems are now in operation..?/ In general, it is felt 
that the data available at the State level on vocational education programs are 
insufficient in scope and detail to meet recent expectations for evaluation to 
display program effectiveness. 

State Vocational Education Program Area Units. Specific program groups 
within the State departments of vocational education are responsible for evalua- 
tion of programs within their areas. State supervisors or directors of areas such 
as consumer and homemaking education, cooperative programs, etc., are responsible 
for development and initiation of evaluation activities, as well as for providing 
assistance to local programs in their areas in developing evaluation strategies 
and conducting the actual evaluations. They are further responsible for assem- 
bling evaluation data and results at the State level for coordination with groups 
such as the RCU for Sta:e Level planning and evaluation purposes. 

The evaluation activities performed at the State level are focused, 
in general, on the examination of program operations in light of the Federal and 
State regulations governing them (as spelled out in the State plan), and in light 
of the stated objectives for each of the programs. The former focus could be 
construed as more of a ’‘quality assurance" activity; the latter could be con- 
strued as broad measurement of program effectiveness. State-level evaluation, 
in terms of methodology, appears to be oriented toward examination of program 
outcomes in light of stated objectives, rather than toward comparative evaluation 
or cost-effectiveness evaluation of programs (although State plans indicate that 
evaluation techniques of this nature are being developed). 



^ One notable exception is the "Occupational Training Information System (OTIS)" 
developed at Oklahoma State University. Data published in 1970 suggest that 
the success of graduates is related to program type (public or private) and 
area of training (trade and industry, business, etc.). 
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Data for Plann in g and Evaluation 



The general lack of baseline data useful for planning and evaluation 
is as much a problem at the State level as it is at other levels* For planning 
purposes, data are usually assembled from numerous different sources and 
often are not consistent or compatible in terms of time frame, definitions, 
area of coverage. 

Data now being utlized by States for program planning include the 
following selected types and sources: 

• Data on economically depressed areas— obtained 
from Federal Economic Development Administration 
reports, State departments of urban affair's, State 
departments of labor, reports of private research 
organizations. State economic research divisions 

• Data on labor market needs-obtained from reports 
of labor demands and projections published by 
State departments of labor, private research organ- 
izations, State CAMPS plans , U.S. Department 

of Labor Manpower Administration 

• Data on population eharacteristics-obtairied from 
reports by State departments of education and 
finance, State employment services, the Manpower 
Report of the President; private research organizations. 

In terms of data on program operations, State-level planners and 
evaluators are limited to the data being reported by the LEAe in their local 
plans and in their annual statistical reports on program operation which are 
needed at the State level for completion of the annual report for USOE. As 
previously mentioned, planners and evaluators also rely on the data made 
available thiough the individual research studies conducted at the State and 
local levels. It shoild be observed, however, that a number of States have 
been devoting much attention to the Development of information systems capable 
of supporting effective planning and evaluation, and are very close to having 
such systems operational. 

LOCAL PLANNING AND EVALUATION 

As a condition of eligibility for Federal funding, the Vocational 
Education Amendments of 1968 require a local plan for vocational education 
Some local plans not only serve as applications for funding but also delineate 
the programs, activities, and services in vocational education. According 
to PL 90-576 local planning (a) is done "in consultation with representatives 
of the educational and training resources available in the area;" (b) should 
programs that will enable the student to make ample progress toward career 



preparation; and (c) in conjunction with the comprehensive area manpower pro- 
gram, should provide solid assurance that the vocational education needs 
within that community will be served. 

Rased on the provisions set forth in PL 90- j76,each State develops 
guidelines for a vocational education master plan which local areas use in de- 
veloping their own plans. Although there is little consistency in the way 
guidelines are developed, the following is a representative "shopping-list" 
of general items that go huo a local plan: 

• The basic philosophy and major objectives of the 
school district 

» An analytical presentation of the neecs, interests, 
and abilities of all students to be served in the area, 
including the disadvantaged, handicapped, dropouts, 
etc . 

• An analytical presentation of community needs and 
opportunities, including projections of future trends 

• The educational and guidance programs currently 
operating in the district, and those :hat will be 
needed in the future based on trend projections 

o The data and additional information necessary for 
implementing programs (i.e., research, curriculum 
development of guidance services to include place- 
ment and follow-up, teachei recruitment and educa- 
tion, and any needed organizational changes) 

• Strategy for community involvement and the develop- 
ment of public information services 

• Probable program costs and fundinc strategies with 

a description of how Federal, State, and ocal funds 
will be allocated 

• A program evaluation plan and methods that might be 
used to implement needed changes in program con- 
tent, etc. 

• The sequence of steps to be followed, 

A few localities have relied heavily on i general advisory committee 
in the formulation of a vocational education master plan. The advisory com- 
mittee includes facility, a Jministrative personnel, and representatives from 
the local community. For example, organizations outside the public school 
system that have sometimes been represented inchide OEO, Urban League, 
State employment service, labor council, mayor's o r fice, CAMPS commission, 
public health department, human rights comnr'.ss ion , and private schools. 



In addition to the genera! advisory committee, a master plan steering committee 
may also be forp\ed to further contribute to overall planning, and the steering 
committee may be augmented by additional administrator, or, in some cases, 
students, and supported by specialists from the general advisory committee. 

Joint planning and coordination of programs occur at this level. 

At least two different subcommittees occasionally function below the 
steering committee. A curriculum committee may be charged with the i c -si- 
bility for approving new subject matter cr course changes. In some iustw 
advisory committees are also formed for each vocational education program. 

Their activities include the development of guidelines with respect 10 the 
following tasks: 

• Identify needs for vocational training. 

• Prepared job descriptions. 

• Prepare task analysis. 

• Develop currlculums . 

• Develop course content . 

• Provide work experience » 

a Determine need for equipment and facilities. 

• Determine criteria for selecting students. 

• Determine methods for recruiting students. 

• Review, evaluate, and suggest curriculum changes. 

It should be observed that the effectiveness of any advisory committee 
or subcommittee Is a function of the support given it by the top local school 
administrator. The effectiveness of the local plan in general is dependent 
upon this administrator's willingness to respond to expert recommendations 
and redirect ex isting resources to meet the vocational education needs identified. 

Some of the more progressive localities are beginning to move toward 
a "learning systems" approach in vocational education planning. Functions 
are identified and then grouped into meaningful combinations i:o facilitate the 
implementation of the plan. For example, the functions are sometimes com- 
bined in the following ways: a) data collection-popuia ion analysisjob market 
analysis/job performance requirements analysis; (b) occupational guidance 
and counseling-occupational education promotion/student red uitrnent/guidarce 
and counseling services/placement; (c) occupational inst r uction-curriculum 
resources and ancillary services/program planniny/program review/cccupational 
instruction; and (d) evaluation. 

Data Collection Fun ction 

Population Needs Analysi s. The major source of data is a current in- 
formation file, maintained and updated on a yearly basis, which describes the 
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in- school and out-of-school population to 3 e served by vocational education 
programs . Included in the file are regular publications and informal memoranda 
received by agencies such as the Stale employment service, Model Cities, the 
Human Rights Commission, the local CAMIS organization, and the public school 
system. Other sources of information can be identified and collected as needed, 

l ob Market Analysis. A second f.le, similar to the population needs 
fi e, is used to maintain information on tha local job market. Data from the 
file are used for planning and assessing reed for vocational programs offered 
by the school system. In addition to the data sources used for the population 
naeds analysis file, periodic reports frorr the city planning department, the 
local chamber of commerce, and the U.S, Departments of Labor and Commerce 
ere collected, compiled, and analyzed 0:1 a continuous basis. 

lob Performance Requirements Analysis. In collaboration with the local 
community* a task analysis is performed o determine job specifications for each 
occupational program. Data for the analysis are furnished by knowledgeable 
individuals who understand what skills are required for the job and know what 
is needed for certification. This analy sis would apply not only to training for 
entry level employment, but to advanced training and preparation to meet pro- 
motional criteria as well. Standards o performance related to each program 
are established and revised as required. 

Occupational Guidcx nee and Co m ^sel inf [ 

Occupational Educa ti on Pro m otion. One duty of the administrative 
staff and faculty is to promote (on a regular basis) the merits and advantages 
of the occupational programs offered by the school system. Communications 
with the general public and appropria o institutions are to be maintained 
through various formal and informal nredia. Important targets for promotional 
material are feeder schools, industrial and labor groups, and the business and 
professional communities. Because his is not a funded activity at the local 
level, the amount of effort that is ac ually being expended in this area is 
uncertain . 



Student Rec ruit ment . Administrators and faculty are often assigned 
some responsibility for student recru tmerit. The school district relies heavily 
on personal contacts between school personnel and the im-school and out-of- 
school population to effect a satisfac tory enrollment in vocational education 
programs. Classroom visits by potei tlal students or guidance counselors are 
sometimes an effective method of drawing attention to specific courses of 
instruction. 

Guidance and Couns el ing . \ critical but often weak link in the system 
is the occupational and career guidarce service. Ideally, counselors and mem- 
bers of advisory committees perform ihree functions: they assist students in 
making sound career choices based o 1 interests, abilities, and aspirations; 
they provide continuous assessment of students with respect to performance, 
progress, and career direction; and tl ey keep students informed about oppor- 
tunities and conditions of the Job mur ;et . 
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Placemen t . The public school administration and faculty have respon- 
sibility for providing placement assistance to students enrolled in vocational 
education piograms. Included in this package are current information on the 
job market and employment opportunities; assistance in finding employment or 
entry into advanced training programs; and needed encouragement for students 
trying to find job opportunities on their ov/n initiative. It appears that place- 
ment services provided by the public schools are relatively unstructured and 
informal in comparison to what is offered by the local employment services 
offices/ which view job placement as a major responsibility . 

Occupationa I. In struction 

Curri c ulum Resources and Ancillary Services. The faculty and admin- 
istration are charged with the responsibility for identifying and evaluating re- 
quired resources such as equipment, materials, teaching aids, physical plant, 
special instructors, and noncertified personnel. They also provide ancillary 
services: in-service education for teachers, libraries, and audiovisual and 
model shops to support vocational programs and services. Evaluation results 
are reported regularly and included in an annual report for review by admin- 
istrative personnel and for budgetary considerations. 

Program Pfenning. Program planning is done annually and includes 
5-year projections that are judged to be realistic goals and in the best interest 
of the community. The planning committee , mentioned earlier, sometimes in- 
cludes ariywhere from 20-50 individuals representing administration and faculty, 
business and industry, community organizations, school students, etc. How 
often the committee meets is up to the superintendent of schools; most meet 
at the beginning of the fall term to consider the budget and at least twice more 
during the course of the year. In some localities, minutes of the committee 
meetings, rather than a formal report, are u sea to support future changes in 
program content. 

Data used for planning come from a variety of sources but mainly from 
the information files established from the population and job market analyses. 
Although it should be a sine qua non for all planning, only a very few districts 
have comparable cost/ student-hour data for vocational and gener l education. 
Some districts either already have, or are in the process of chancing over to, 
o PPB system; others seem to be making little progress iri that direction. 

Program Review . The quality, completeness, and coordination of voca- 
tional education is to be examined by a program review committee of school 
district faculty ar i administration, management leaders, union representatives, 
and concerned parents and students* The review is designed to provide an es- 
timate of how well programs are meeting population and labor market needs. 

It is not certain at the Federal level whether the criteria used by these com- 
mittees are adequate to measure the effectiveness of programs in meeting 
these needs . 
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Occupational Instruction. The goals set for occupational instruction 
occasionally include the following: it should be consistent with the readiness, 
interests, aspirations, and abilities of the individual student; it should bo 
offered at appropriate times in order to meet target group needs; preparatory 
training, upgrading, related t aining, retraining, and refresher training should 
each be of appropriate length and duration; instruction should include lecture- 
demonstration presentations , practical applications exorcises, and work-study 
experiences in appropriate areas; and instruction should be provided by compe- 
tent and qualified teachers. 

Evalua tion 

Evaluation o i: vocational education programs, like the program planning 
function, varies widely from area to area. In some planning documents, page 
after page is devoted to specific questions to he asked; in others, evaluation 
procedures merit a scant line or two. It would probably be safe to say, in 
general, that techniques of evaluation are lagging far behind other areas of 
planning methodology. AL times, evaluation has been performed by committees 
or specified individuals within the school system. A few districts have asked 
the State for help or have looked for assistance from private associations of 
schools and colleges. Local evaluation is apparently process-oi ien .ed in 
general, but follows the traditional accreditation approach to evaluation. 

At least three types of evaluation have been done in more progressive 
areas. The first type is preprogram evaluation, which utilizes data on items 
such as instructional objectives, area employment needs and opportunities, 
persons to be served b/ the programs, criteria for selecting qualified personnel, 
and evidence that former evaluation results are being used in current or future 
planning. A second category is ongoing evaluation, that questions how well 
each basic function is meeting its objectives. Required here are data on 
methods, materials, and other program components leading to modification or 
drastic changes in practices one procedures. The third category is end-point 
evaluation, which may use follow-up studies on graduates and dropouts. Cases 
may be based only on teacher judgments, At other times, data for end-point 
evaluation have been obtained via postcard questionnaires mailed to both June 
and January graduates. The data utilized include, for example, number of 
placements, location of placements, and how long on the job. Other kinds of 
data that might be useo at various limes include enrollment levels, enrollment 
attrition, number of persons returning for training after completing certain 
phases of the Gaining and, in a few instances, employer satisfaction with 
trainees , 



In summery, local planning varies considerably from area to area — even 
within the same State. Some plans are very comprehensive while others are 
poorly formulated presentations of statistical data written for the sole purpose 
of complying with State regulations. The design, development, and implemen- 
tation of local vocational education plans probably depend on the standards of 
performance that exist among the individuals charged with the responsibility 
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for the planning. Another factor is the degree of cooperation between the staff 
end faculty representing each division of vocational instruction— secondary , 
post-secondary, and adult. 

A few more enlightened areas have insisted on joint advisory committees 
to facilitate greater coord jrrtion and consistency among divisions. Other com- 
munities are moving toward regional planning in an effort to combat problems 
affecting districts linked by common boundaries, such as duplication of occupa- 
tional programs, facilities, equipment, and personnel, and poor varget group 
coverage . 

It is feit at the Federal level that, in general, local education agencies 
are not yet getting the analysis of data that would be necessary to weigh alter - 
native programs and make well-grounded recommendations for local programming. 
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APPENDIX D 

A NEW APPROACH TO VOCATIONAL EDUCATION: FOCUSING PROGRAM 
OPPORTUNITIES ON THE HIGH-POTENTIAL POPULATION AS A 
POSSIBLE POLICY ALTERNATIVE 



BACKGROUND 

Complaints about vocational education are more and more frequently 
expressed. It is alleged to be quite costly and not very effective. Although 
there is no doubt that substantial improvements are possible in the current 
organizational modes and delivery mechanisms for vocational education 
services — e.g. , better teachers and equipment, more attention to placemeni, 
use of private facilities — it may also be the case that organization and delivery 
are not keyed to fundamental trends in the U.S. economy and in American so- 
ciety in general. Certain features of current social reality would seem in fact 
to be at variance with the structure of contemporary vocational education and 
with our expectations about it. Thus this appendix is provided to supplement 
the Volume I discussion of vocational education structure by exploring some 
of the discontinuities and inconsistencies in that structure and attempting 
to derive directions for change. 

In 1958 there were more than 7.5 million vocational-technical 
education students in the United States (see Table 12). More than 8 million 
people were enrolled in federally operated, aided, or regulated programs to 
provide vocational skillo. This figure was nu, than 70 percent higher than 
the number of persons enrolled only 4 years earlier. Table 13 gives per- 
centage breakdowns by age group for combinations of programs for the years 
1966 and 1968. Vocational education, strictly defined, is seen to concen- 
trate on the secondary school age group (15 to 18), with 6 lesser concentration 
on adults (defined as 22 years old and over). The post-secondary age group 
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TABLE 12.— Persons enrolled in federally operated , aided, and 
regulated programs that provide vocational, technical and 
skill training, by age group, for years 1964, 1966, 1968. 
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U,S. Department of Labor, Manpower Report of the President (1970), U.S. Government Printing Office, Washington, 
pp. 300, 310, 312. 316, 321 , 322. 



(19 to 21) accounts for only a small fraction of enrolled students, but the size 
of this qroup has expanded in absolute terms by a factor of 3 since 1964, when 
it accounted for only 4 percent of vocational educational enrollments , Other 
forms of federally funded skill training are seen to be more eveniy balanced 
by age group, with something like a third in each group in both 1966 and 1968. 
Overall, in 1968 about half of all vocational program enrollees were under 19 
and approximately 6 out of 10 were under 22. 



TABLE 13.— Percentage of persons in various age group categories: enrollees 
in federally operated and aided programs offering vocational training, 

1966 and 1968 . 
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All 
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42 
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Out-of-School 


100 
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33 
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35 



~^The 1964 percentages for the three age groups were 47 percent, 4 per- 
cent, and 4C percent - 

Source: Table 12. 



If sorr.j practioners and observers had their way the concentration 
would shift to even younger age groups through the extension of intensive vo- 
cational preparation to children below high school ag e . 1/ It is offered as 
a remedy for the apparently disappointing effectiveness of current vocational 
education, discussed in other papers- submitted to this task force. The usual 
defense of the view rests on the implicit argument that if a program lacks im- 
pact, expanding the program is the solution. Often accompanying this argu- 
ment is the observation that other countries— West Germany, USSR, Switzer- 
land— have extensive vocational education/apprenticeship systems that 
supply trained and work-ready 15 -year-olds to the labor market 



“^See, for example, V. A, Adams, "Vocational Training; Still for Someone Else's 
Children," School Management , September 1970, pp . 12-15. 

2 / 

— For a general review of research o n youth behavior in many countries, see 
L. Rosenmayr, "Towards an Ovcr/iew of Youth Sociology," International 
Social Sc ien ce Jou r nal , Vol. 2, No. 2, 1968, pp, 286-315. 
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To many, programs like these aimed at early teenage years are advantageous 
because they help meet skill shortages, provide young people with constructive 
activities, anc , in general, seem to fulfill desires for orderly, controlled social 
relationships. But in many respects they speak more to conditions of the societies 
where they were developed than to the society for which they are bemg advocated. 

In the United States it seems to be the case that the age at which people 
emerge from dependency into true self-sufficiency is systematically receding. 

For the sons and daughters of the upper middle class and for many college 
students of diverse backgrounds, formal education continues to and often past 
the mid-twenties. Growing numbers follow formal education with a year or two 
of travel (and man r , of course, plan permanent dropping-out). Evidence from 
the most recent Manpower Report of the Presiden t 1/ shows lhat the fraction of 
young people employed in jobs has been trending downward over time (see 
Figure 3 }. 

The figure treats as not employed those young people who were either not 
in the labor force or who were searching for but without jobs. This is because 
for young workers the distinction between non-participation in the labor market 
and unemployment is not very clear. In other words, when a mature family man 
claims to be seeking a job (i.e. , in the labor force but unemployed) his claim 
carries more credibility than the same statement by a teenager. In any case, at 
least some officially unemployed young persons are probably not diligently pur- 
suing jobs. Further bits of evidence as to the weak labor force attachment of 
young workers are easy enough to find. In 1969, for examply, the unemployed 
young worker (16-19 years cld) was almost as likely to have left his last job as 
to have lost it; 12 percent of the unemployed of this age left jobs; 15 percent 
lost them. For mature male workers, on me oiner nanu, leaving a jcd is a 
much less likely reason for unemployment than is losing a job (17 percent 
versus 58 percent) . 4 / 

For young males the labor force participation rates, even of those not 
enrolled in school, has been declining through the 1950s and 1960s as shown 
in Table 24. A similar, although not so pronounced, trend has been noted for 
young female workers ( in Manpower Report j ^f the President for 1970, cited 
previously, page 250) . 

The explanation of these phenomena is no doubt complex. Partly they are 
the result of affluence; parents use increased Incomes to purchase more leisure 
(and more career preparation) for their offspring. Added to this, probably — 
although here the statistical evidence is weak — is a reduction in the demand for 
unskilled labor as the economy grows in technological complexity. The shift 
in general attitudes concerning the importance of work and leisure is also of un- 
doubted importance, although harder to pin down empirically. A societal drift 
along the continuum from Puritanism to hedonism impacts most strongly on the 
young, but it also influences the way parents view norremunor ^tive activity by 
their children. 
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Figure 3.— Ratio of employed persons to total i 

1952-1969. 



'onrce: U.S. Department of l^abor, Manpower Report of 
U.S. Government Printing Office, Washington, 
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18- to 19-year-old males 



16- to 17-year-old males 
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the Presiden t (1968), 
D. C. , pp. 224, 229. 
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TABLE 14.— Labor force participation rates for young males not enrolled 
in school, 1953-1968— selected years 



Year 


Age group 


16-17 


18-19 


20-24 


1953 


86.5% 


95.9% 


96.1% 


1959 


80.1 


92.9 


96.8 


1964 


72.5 


92.0 


96.6 


1968 


71.1 


87.8 


94.2 



Source: U.S. Department of Labor, Manpower Report of the 
President (1970), U.S. Goverment Printing Office, 
Washington, D.C., p. 250. 



Nor are parents alone in accepting, and even encouraging, later and later 
entry into the labor force. In 1955 about 28 percent of people in the 18-21 
age group were in institutions of higher education; by 1969 this fraction had 
grown to 44 percent. In this same period graduate enrollments increased by 
better than a factor of 3 , i . e . , from 251,000 to 79 8 , 000 .H/ Governmental de- 
cisions and action had something to do with this increase, For example, Federal- 
State-local support of higher education grew from just over $1 billion to almost 
$9 billion (of constant purchasing power) between 1950 and 1969. (See Table 15) . 
Not only has t;he magnitude of resources grown, but the level of support per student 
has shot upward as well. In 1950 the average higher education student was sub- 
sidized with about $460 in public funds; by 1969 the subsidy had grown to $1,240 
(both figures are in 1957-59 dollars) . 

Increased communication, through the media and the propinquity of people 
in urban settings, has meant that the life style adopted by college students has 
become increasingly apparent ana Increasingly available to noncollege young 
people of similar age. (Witness the "street people’ colonies in Berkeley, 

Madison, Cambridge, and other college towns.) Woodstock and the counter- 
culture appear to be age-specific but not very class-specific. 

All c:f this suggests that In the America of the last third of the twentieth 
century, the people who will occupy the kinds of positions vocational training 
prepares for will not be ready to assume those positions until perhaps they are 
well into their twenties . And this suggests in turn that vocational education 



~ American Council on Edu ation, A Factb o ok on Higher Education , 
Washington, D.C., 1970, p. 2. 
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opportunities should be focused on the years immediately ^-receding serious 
commitment to the labor force and to a career. When vocational education op- 
portunities are offered too early the benefits to stuc snts are probably reduced: 
absorption of material is impeded; attention wander: ; inappropriate courses are 
pursued; skills, even if attained, are dissipated in ;he interval that precedes 
settling down . Thus, it is the contention here that premature emphasis, rather 
than generally insufficient resources, lies behind the problem of ineffectiveness 
in vocational preparation programs. ■ 

If this charge is true it. should cause us to reexamine the changes in voc- 
ational training strategies and policies for the 10 critical years of a young 
person's life— from age 15 to age 25. The final section of this appendix sets 
forth soma possible guidelines for an improved system of vocational education 
built upon current American social and economic reality. First, however, it is 
useful to review in a more systematic fashion the evidence on the relationship 
between the effectiveness of vocational education and the chronological age 
of the trainee . * 

f 



TABLE ). 5 • Per student contributions to higher education by government, in 
dollars of constant value, 1350-1959 —selected years 





Government contributions ($ billions)^ 


Year 


No. students 
(millions) 


Federal 


F 

State l: local 


Total 


Contribution 
per student ($) 


1950 


2.6 


0.6 


0.6 


1.2 


460 


1955 


2.7 


0.5 


1 .0 


1.5 


560 


1960 


3.6 


1.0 


1.5 


2.5 


690 


1965 


5 .5 


2.6 


3,: 2 


5.8 


1 ,050 


1969 


7.1 


3.8 


5.0 


8.8 


1,240 



- In dollars of 1957-59 purchasing power according to the Consumer Price 
Tndex of the Bureau of Labor Statistics . 

Source; American Council on Education, U,S. Department of Commerce, 
1970. 
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EFFECTIVENESS OF VOCATIONAL EDUCATION AND THE AGE OF THE TRAINEE 



A recent study compared the economic benefits —wages , earnings, 
employment stability— of a sample of more than 2,000 vocational course gradu- 
ates in an attempt to dicover, among other things, the net contribution of level 
of schooling to economic bene fits. J>/ The levels identified were high school, 
pest-secondary (includes all noncollegiate courses), and junior college. It was 
discovered that when the characteristics of students and labor market conditions 
were held constant, people who had junior college training invariably did better 
than secondary and post-secondary students. Post-secondary students tended to 
do better than secondary vocational students oy lesser amounts, although the 
difference often was not significant. U An earlier multivariate analysis of a 
smaller group of North Carolina vocational course graduates showed strong net 
positive effects of post-secondary training {as compared to secondary) on monthly 
e r rnings.— - ' Interesting background to these studies is provided in Corazzini, 

1967; Kaufman and Schaefer, 1967; Kaufman, et_al. , 1963; and Sharp and Myint , 

1969. None of these studies, however, clearly answers the question on critical 
age. Having other objectives, the authors did not consider it important to analyze 
secondary versus post-secondary courses (whether collegiate or not) as alter- 
natives , and therefore they aid net employ as a variable the age at which vocational 
education was absorbed. The people in their samples who had post-secondary 
vocational education experience might also have had secondary courses in vocational 
education. Therefore, it cannot be determined whether the coefficient on level of 
schooling measures the contribution of an additional exposure to vocational edu- 
cation of a substitute exposure at a later date. 



— S.F. Fernbach, "An Ai 'lysis of the Economic Benefits of Vocational Education 
at the Secondary, Post-Secondary and Junior College Levels," M.A. thesis 
(Industrial Relations) , University of Wisconsin , Madison, 1970. 

p. 97. 

8 / 

— A.B. Carroll and L.A. Ihnen, "Costs and Returns for Two Years oi Post- 
secondary Schooling: A Pilot Study," Journal of Polit i cal Econ omy, 

December 1967, p. 862. 

9/ 

— A.J. Corazzini, "When Should Vocational Education Begin?" Center for 
Studies in Vocational and Technical Education, University of Wisconsin, 
Madison, 1967; J.W. Kaufman and C.J. Schaefer, P repa ration of Yout h 
f or Effective Occupational U tili zation: The Role of Secondary Schoo Is in 
the Preparation of Youth for E mployment (final report, Project No. OE-2897) , 
Institute for Research on Human Resources, Pennsylvania State University, 
1967; KW. Kaufman, et_jal. , A Cos t- Effe ct iveness Study of Vocationa l 
Education, A _C omparison of Vocational and Non-Voccitional Ed uca tion in 
Secondary Schools , (final report, Project No. OE-S12) , Institute for 
Research on Human Resources# Pennsylvania State University, February 1968; 
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Sophisticated analyses would be required to test adequately the rela- 
tion between training effectiveness and age at which training is received and, 
given the paucity of processed data and the innppropriateness of existing stu- 
dies, it was not possible to accomplish such analyses for this presentation . 
Longitudinal data are necessary, that follow a person from training well into 
his working life. This kind of data does exist, but it has not as yet been 
organised to help answer questions about the critical age for vocational edu- 
cation . For example, since 1960, Project TALENT has collected materials on 
the courses followed by a sample of high school students that originally num- 
bered -100,000. Flanagan {1962) reports follow-up interviews by mail at 1-, 

S-, and 10-year intervals after high school graduation. 10/ The "Specialty 
Oriented Student" (SOS) surveys interview people in post-secondary institutions 
(colleciate, public, private) and gathers information on training experience and 
post-program joh success 6 months, 2 years and 5 years later (see Hoyt, 1968).—' 
A comparative analysis of these two sources, which standardized for the char- 
acteristics of the students and the quality and type of training, should permit con- 
clusions as to the relative efficacy of teenage as compared to delayed training. 



11 / 



The preponderance of evidence seems to suggest the conclusion that 
vocational education experienced in early maturity, around the ages 23 to 25, 
might give more prornire of leading to meaningful improvement in lifetime economic 
prospects. Junior and community college experience appears to confer moie bene- 
fits than high school vocational courses and, although the comparisons are dif- 
ficult to make, post-college training under bolh public and private auspieces 
might have an even higher payoff. These bene fit estimates refer, essentially, to 
yeatiy income. They do not take into account two important negative aspects of 
delayed vocational training: the fewer years o ' active worklife that would remain 
to old ar students, and the higher costs in terms of foregone income whLch they 
suffer as a result of not working at, say, age 23 rather than age 18. 



According to the most recent estimates the mean yearly income of a full- 
time worker in the age group currently favored in vocational education programs 
(i . e . , J 6-19) was $2,994; his counterpart in he age group being considered 



L.M. Sharp and T. Myint, "Graduates of V< cational-Technical Programs in 
Jun or Colleges: Results of a Follcw-Up St idy of the Class of 1966" (pro 
pared for the Center for Studies in Vocational and Technical Education, 
University of Wisconsin) , Bureau of Applie< Social Research, Washington, 
D.G., 1969. 

-^^See J.C. Flanagan et al . . Design for a St\ dy of American Youth , Houghton 
Mifflin, Boston, 196Z. 

— ^K.B. Hoyt, "The Specialty Oriented Student Research Program: A Five-Year 
Report," Vocation a l Guidance Quarterly , March 19 68, pp . 169 -176. 
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1 2 / 

for concentration (i.e., 20-24) earned $5,574 per year. — Assuming that a voc- 
ational course for each would take 1 year, that the direct costs of education (i.e. , 
teachers, buildings, equipment) would be the same, and that each would retire 
at age 65, then at a discount rate of 5 percent the person receiving training at 
23 would have to earn an average of at least $150 more per year to justify the 
higher " foregone-earnings" cost of training and a shorter work life. At a dis- 
count rate of 10 percent the person with delayed training would have to earn about 
$260 more per year. This may be compared with the dollar premium in earnings 
Fernbach derived for junior college and post-secondary students as compared to 
high school vocational students. — / While noting the earlier caveats as to the 
relevance of her findings for the purpose here, we see that the differentials were, 
respectively $198 and $83 per month . 

If the social trends noted in the first part of this appendix are proceeding 
apace, then the Immediate post-adolescent years are increasingly characterized by 
low earnings, sporadic attachment to the labor force, and a generally experimental 
attitude toward jobs, 14 / These suggest in turn a declining receptivity to skill 
training and perhaps a growing nap between the payoffs available from early and 
late training as described hen. 

Clearly, the most important consideration in deciding the preferred 
age to offer vocational education depends on when "career readiness' 1 begins 
for potential workers. Data on this topic are not readily available. One po- 
tential source, however, lies in the Social Security Administration's "Continuous 
Work History Sample," which has been organized into longitudinal records on 
employment and earnings fcr almost 900,000 covered workeio for the period 
1957-67 (LEED file). Workers are identified by age, race, sex; and quarterly 
information on earnings, firm and industry affiliation, and work location is 
given* Thus it is entirely possible to analyze these data to elicit the relation- 
ship between work force behavior— whV.h might be defined, say, in terms of 
geographical, firm, and industrial mobility, the rate of earnings increases, and 
the like — and age of the worker or the age at which he entered the full-time 
labor force* Although such analyses hrwe not as yet been performed, itis hypoth- 
esized that the data would show that "settling down" behavio r is a function of 
age, and it might even appear that workers who enter later have more stable 
employment histories , Both points, however, are mere conjectures until such 
analyses are performed. 

17 / 

— U.S. Department of Commerce, Current Popu lati on Repor ts . Consumer In- 
come , U.S. Government Printing Ofiice , Washington, D.C, , December 1969, 
p. 90. 

13 / 

— Fernbach. op. cit . 

— / SJ. Carroll and A. H. Pascal, Youth and Work: Toward a Model of Lifetim e 
Economic Pros pects (RM-S891-OEO) , the RAND Corporation, April 1969. 
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The link between behavioral stability and age extends, of course, to 
areas of life other than woik. l^/The sociological and psychological literature 
on delinquency (Singell, 1967; Glaser and Rice, * J 59) ,1$/ mobility (U.S. Depart- 
ment of Commerce, 1970), H / drug addiction (Sagi and Weliford, 1968), and 
even on alcoholism (Alexander, 1967) ,-12/auto accidents and neuroses {Shepard 
and Greenberg, 195 7) tend to show that less deviant patterns begin to emerge 

in the mid-twenties. Even the trend of decline in average age of marriage that 
occurred in the post-war years has begun to dissipate .-£1/ All of this seems tc be 
added evidence for considering the years between about 23 and 25 as a strategic 
period for the provision of high quality vocational education programs . 

The arguments advanced in this paper in favor of delay in offering 
comprehensive vocational education would appear to apply both to young men 
and to young women, although much of the evidence seems to pertain to males 
rather than females. Indications of deviancy seem to diminish and the tendency 
to settle down seems to emerge in a more marked fashion for young men at these 
ages. In work-oriented behavior as in many other forms of behavior, females 
seems to mature earlier. 



In general, see; B. Berger, "Adolescence and Beyond; A Review of Three Books 
on the Problems of Growing Up," Social Fo ices, October 1963, pp. 394-408. 
R.E. Grinder, "Distinctiveness and Thrust in American Youth Culture," Journal 
of Social Issues , April 1968, pp. 7-20. 

^1. D. Singell, "An Examination of the Empirical Relationship Between Un- 
employment and Juvenile Delinquency," Am erican Journ al of Economics 
and Sociology , October 1967, pp. 377-385; D. Glaser and K. Rice, "Crime, 
Age and Employment," American So ciological Review, October 1959. dd.679- 
686 . 

17/ 

U.S. Department of Commerce, Statistical A bstract o f the United States . 

U.S. Government Printing Office, Washington, D.C.~ 19/0. 

18 / 

P.C. Sagi and C.F. Weliford, "Age Composition and Patterns of Change in 
Criminal Statistics," Journal of Criminal L a w, Criminology and Police Science , 
March 1968, pp. 29-36. 

19/ 

— C ,N . Alexander, Jr., "Alcohol and Adolescent Rebellion," Social Forces , 

June 1967, pp. 542-55C. 

9 0 / 

M. Sheperd and E.G. Greenberg, "The Age for Neuroses M illbank 
Memorial Fund Quarterly. Tulv 1957 , pp. 258-265. 

23/ 

J.R. Rile, "Trends and Differentials in American Age at Marriage," Millbank 
Memorial Fund Quart€ rly t Apr!) 1965, pp. 219-234. 
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A VOCATIONAL EDUCATION PROGRAM FOR TODAY'S LABOR MARKET 



The implication of the foregoing arguments and evidence is that 
vocational education opportunities should locus more precisely cn a group 
slightly older than the current target population, which consists mostly 
of high school and junior college age groups, i.e., those under 20 years 
of age. This does not mean that the training needs of the younger group 
should be Ignored, but does suggest that a change in emphasis would be 
salutary. A system that would seem to fit this view of the requirements 
would be one that of:ers counseling, work exposure and experience, and 
education in basic skills and cultural and rjcreational pursuits to people 
in their late teens and early twenties, while providing intensive training 
in technical institutes, to more mature young people. 

For the younger age group high schools and community colleges 
might present a pre-vocational program that consists of classroom work 
combined with part-time and summer jobs. The classroom instruction 
would concentrate on some of the more general industrial and commercial 
skills while at the same time offering the pre-vocational student a menu 
of courses designed to enrich his cultural perspectives and his recreational 
possibilities. An active counseling and job placement program would also 
be provided by the schools. Counselors woulu attempt to work out a program 
of job experience for their students, structured to provide them with a series 
of controlled experiences in the world of work. The stress would be on pro- 
viding information and opening vistas rather than on what now often amount to 
prep’.ature attempts to insert a young person into a permanent career slot. In- 
tensifying the payoff in learning experience might well result in frequent job 
changes by the student and thus, perhaps - in the general appearance cf insta- 
bility. In order to overcome any employer reluctance to offer part-time and 
shoii-term opportunities it may become necessary to provide small subsidies 
to eligible working students, to bridge the gap between their value to (and 
wages from) an employer and their own requirements for living expenses. 

The pre-vocationalists ough* not to be heavily penalized in loss of 
services if they drop out of the classroom learning part of the program. As 
long as the counseling contact is maintained, job finding and job placement 
sendee snould continue, and in some cases the wage subsidy should continue 
as well. The main requirement for participation in the n ew vocational education 
program ought to be a serious commitment to useful exploration of alternative 
career possibilities, not good attendance at classroom sessions. 

High quality technical training opportunities would be available 
to those who complete the pre-vocational phase described but may be open 
as well to applicants admitted on other criteria. There are two basic models 
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under which the technical training might be provided. Each has merits, and 
the wisest: course would be to organize demonstrations of both approaches 
and to accompany each with a systematic evaluation of its costs and benefits. 

The first model involves the organization of technical institutes under 
public allspices. They would depart from the current MDTA institutional 
training programs by concentrating on a somewhat older cross section of 
students, by offering a higher quality of training in somewhat more demanding 
skill categories, and by providing stipends to students sufficient for modest 
family support. The quality of training should be at least as high as that offered 
in the Armed Forces. In time, certificates of completion offered by the tech- 
nical institutes, if quality of training standards are maintained, would come 
to have substantial value in labor markets. The issuance of a certificate of 
competency in a technical skill by a respected Government agency—with the 
backup commitment of resources implied by such a procedure— would begin co 
redress the balance in public attention to the problems of college and non- 
college groups . 

The second model would make use of voucher schemes for the delivery 
of vocaticnal training services to the eligible population. Trainees would in 
effect receive tuition grants in the same way that National Merit Scholarship 
students do. Grants or loans for living expenses, which might depend on family 
size and other sources of income, would accompany the "scholarships." Vo- 
cational training services for the age group being discussed seem a very appro- 
priate application of the voucher method for a number of reasons. Quality 
control over services providers does not present large problems since the ef- 
fectiveness of the product is directly observable, with an admitted time lag, in 
the employment success of graduated trainees. And the voucher recipients 
would certainly have strong incentives to seek out the best trade schools at 
which to use their tuition grants. Thus both supply and demand conditions 
seem such as to presage a successful use of this service delivery mechanism. 
The experience with educational benefits under the GI Biii following World War 
II, which was rather similar to what is proposed here, suggests that the stra- 
tegy is workable. In any case, a carefully evaluated demonstration program 
should reveal whether the voucher mechanism is promising. 
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APPENDIX E 

THE "MARKETABLE SKILL” CONCEPT 



CHARACTERISTICS OF A MARKETABLE SKILL 



If a vocational education program is to attract enrollees, a marketable 
skill must have private (emollee) benefits which cover its private costs. This 
does not necessarily mean that a marketable skill is a skill which yields the 
highest private rate of return. It simply must have a nonzero rate of return or 
must have a positive net present value when private costs and benefits are dis- 
counted at the relevant private interest rate. Thus, the provision of a market- 
able skill does not necessarily ensure an economically efficient investment from 
the priv ate standpoint. There can be alternative users for private funds, in- 
cluding investment in other types of education (very generally defined), that 
yield a higher rate of return or a higher nut present value. 

Because of the subsidy element involved in public education, the pro- 
vision of a marketable skill could imply a positive private rate of return while the 
the. social . ate of return could be zero or negative. Thus, there is no guarantee 
that a prog. Jm is economically efficient from a social standpoint lust because 
its graduates are hired. And, to reiterate, the fact that a person is hired, even 
steadily employed, is no guarantee that the skill in question gives the most 
advantageous private return-but it is marketable. 
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THE CORRESPONDENCE BETWEEN " MARKETA P LE 1 ' AND "SHORT SUPPLY' 



A marketable skill could be a skill which is in short supply. Presumably, 
the best evidence that a skill is marketable is demand for it by a firm or industry. 
However, there are two types, or concepts, of "shortage" which should be s- 
cussed here — structural and nonstructural. Figure 4 permits comparison of the two. 





Quantity of Ipbor/unit of time Quantity of !jOor/unit of time 

Skill A - structural shortage Skill B - nonstructural shortage 

Figure 4 . — Structura 1 versus nonstructural skill shortage. 



A structural bottleneck in Skill A exists, since the supply curve for the 
skill becomes vertical once the quantity Oq i is brought forth. Beyond this point 
wage increases will not induce a further increase in quantity supplied. Voca- 
tional education could be used to increase the supply of Skill A, either by 
moving the supply curve to the right or by changing its shape. The distance 
W^Vv ’0 represents a rent being paid t , the current workers in the skill, quantity 
Oq i , The increase in supply of workers in Skill A reduces the private rent paid 
to existing workers but does not affect the incentive to supply labor for this 
original group of laborers. Society, hov r ever, will experience an increase in 
output and may find it efficient to provide this skill training even though some 
income redistribution will occur. An example of a marketable skill category 
characterized by structural shortage would be electricians in New York City, 
or construction workers in Chicago. 

For Skill B, as shown in Figure 4 , firms are originally offering wage 
rate OW , which is less than the equilibrium wage rate. There is a ''shortage” 
of Skill b equal to q q # , but it exists because firms are either unwilling or 
unable to pay the equilibrium wage rate OW e . Under these conditions, at wage 



rate OW^ the quantity of Skill B persons equal to q j q could be "marketed . 11 
Here, again, vocational education would be used to sfiift the supply curve from 
s to s'. This woulo wipe cut the "shortage" and allow the wage rate OW^ to be 
maintained. 

This Latter case does not represent a structural shortage, sin^e by 
offering wage rate OW e / the equilibrium supplied would be achieved at OQe* 

The result of raising the wage rate would be to reduce returns to other factors 
in the industry or to drive some firms out of the industry. In either case, though 
income redistribution would result, this would be more efficient than spending 
additional vocational education funds to abnormally inflate supply. If firms were 
forced out of the industry, this is prima facie evidence that they were inefficient 
firms and that society has better uses elsewhere for the resources tied up in 
them. Also, even though individual workers may receive a positive gross gain 
from being trained in Skill B, a higher net return could be gained for them if 
they were trained in an alternative skill, since the firm or firms in question are 
already operating under high cost, inefficient conditions. 

Occupations with a high turnover rate, but no physical conditions such 
as seasonal factors or changing job locations causing this turnover, are likely 
prospects for skills that either are being paid below equilibrium wage rates or 
that otherwise offer wage rates too low to cover the opportunity costs of holding 
the job. In such cases the complaints by employers of a shortage of workers 
should rot be taken as evidence that the skill is marketable. On the contrary, 
the opposite is implied, likely prospects for inclusion in the false shortage cate- 
gory are such occupations as nurse's aide and waitress. 

SUMMARY 

The notion of a "marketable" skill, while a precise concept on the 
surface, is an ambiguous one in practice. 

o The productivity of a person trained in such a skill 
must equal the money wage rate, but even when 
this criterion is met, the skill may not be the most 
efficient of a set of competing skills . 

♦ What is marketable and/or most efficient in private 
and social terms may differ. 

• Identification of a marketable skill is made difficult 
by problems of identifying occupational needs 

or shortages. 
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